Question Effectiveness Analysis
Aug 24, 2015
Hello All,
I've spent a couple of hours trying to figure out the data Storyline interactions write to our LMS and I can't make heads or tails out of it, I'm wondering if someone could help. We're trying to analyze question and training effectiveness for a few of our courses by analyzing which questions our users get wrong the most often. When using question banks I have no idea how to get back to the question that's being asked.
We get a value that looks like this: "Scene7_QuestionDraw21_Slide6_MultiChoice_0_0" and then the results of that interaction.
Parsing this string I know we're in scene 7 and are looking at "Slide 6" which is a multiple choice question. The problem is that due to the "QuestionDraw", "Slide 6" can mean anything...
Looking at the body of the data, "Slide 6" is sometimes a T/F question or a Multiple Response question...
Is there any way to unwrap this data? Why isn't Storyline's actual slide number not indicated anywhere? Without it the data is useless. What am I missing?
9 Replies
Hi Peter,
Storyline doesn't include slide numbers on the slides or as a part of the output, and the data we send to an LMS is detailed here, but it'll be up to an LMS to decipher and output the data in a way that is readable to you as an admin. You may also want to look at testing the content at SCORM Cloud as it's an industry standard for testing LMS content.
Sorry - I got a little confused about what "Question Draw" meant. I'm assuming this is the question bank? Then within each question bank are the "Slide Numbers" just ordered in the same order that they appear in the bank?
Hi Peter,
The question draw is how you've set up the questions to be drawn from the question bank. You can read a bit more about it here. If you haven't set them to shuffle/draw randomly, they should appear in the order as you entered them.
Thanks again Ashley,
Let me try to rephrase my question. Based on your article on what is sent to the LMS, it sends the Question Draw and the Slide #. Is the Slide # being sent in the order it is in the story or is it the order that it is presented to the user? If it's in the order it's in the story, that's ok, however if it's in the order it's presented to the user - there is no way to get back to the question since there is no other identifying information being sent about the question itself.
Hi Peter, sorry I do not know the answer to the last question, can you use scorm 2004? The question is sent to the LMS in Scorm 2004 so that may help if you can use it.
Thanks Phil. I'll take a look.
Hi Peter,
I did a quick test with LMS debug mode on so I could see what it was sending, and it listed them in the order that you've set - but I also saw the interaction type listed too such as:
Scene1_QuestionDraw11_Slide2_MultiChoice_ so you may also want to look into if your report is showing the question type as well as that should be reported. For my test purposes I used Scorm 1.2 and uploaded to SCORM Cloud. While viewing the debug log, I also had to scroll far to the right in Notepad and I could see the question text too.
Thanks for doing the investigation Ashley. For some reason I'm not seeing the question text with 1.2 - just the answer. But I can always go back and map these now that I know they come back with a static id on the slide number vs. one that's dynamic based on the sort order.
Question Text is not sent in 1.2
This discussion is closed. You can start a new discussion or contact Articulate Support.