Post-quiz Test Item Analysis

Jun 02, 2015

Dear Colleagues, I am evaluating Storyline 2 as the preferred authorware tool for my company. It is important for us to be able to perform post-test analysis and validation on test questions and answers at a macro level. For example, we need to know that a particular question was presented 500 times and that it was answered correctly only 300 times. I understand from the Articulate sales rep that the information necessary for this type of reporting is exported by Storyline 2 and that it is up to our LMS to generate any reports. Does anyone have a sample of the type data that is pushed via SCORM to the LMS? 

 

Thanks,

Callum

6 Replies
Steve Flowers

Hi Callum - 

This will be in the SCORM interactions portion of the SCORM data model. The model varies a bit for SCORM 1.2 and SCORM 2004. For richer data, you'll want to use SCORM 2004 unless your LMS doesn't play well with that version. This is the data reference (what the SCORM file will communicate to the LMS) for SCORM 2004.

Storyline will send the data for each submitted interaction to the LMS using the appropriate elements. It is up to your LMS to provide analytics and reporting capability. What LMS are you using? Some are stronger than others. Many LMS use the labels bolded below for data reports.

  • cmi.interactions._children (id,type,objectives,timestamp,correct_responses,weighting,learner_response,result,latency,description, RO) Listing of supported data model elements
  • cmi.interactions._count (non-negative integer, RO) Current number of interactions being stored by the LMS
  • cmi.interactions.n.id (long_identifier_type (SPM: 4000), RW) Unique label for the interaction
  • cmi.interactions.n.type (“true-false”, “choice”, “fill-in”, “long-fill-in”, “matching”, “performance”, “sequencing”, “likert”, “numeric” or “other”, RW) Which type of interaction is recorded
  • cmi.interactions.n.objectives._count (non-negative integer, RO) Current number of objectives (i.e., objective identifiers) being stored by the LMS for this interaction
  • cmi.interactions.n.objectives.n.id (long_identifier_type (SPM: 4000), RW) Label for objectives associated with the interaction
  • cmi.interactions.n.timestamp (time(second,10,0), RW) Point in time at which the interaction was first made available to the learner for learner interaction and response
  • cmi.interactions.n.correct_responses._count (non-negative integer, RO) Current number of correct responses being stored by the LMS for this interaction
  • cmi.interactions.n.correct_responses.n.pattern (format depends on interaction type, RW) One correct response pattern for the interaction
  • cmi.interactions.n.weighting (real (10,7), RW) Weight given to the interaction relative to other interactions
  • cmi.interactions.n.learner_response (format depends on interaction type, RW) Data generated when a learner responds to an interaction
  • cmi.interactions.n.result (“correct”, “incorrect”, “unanticipated”, “neutral”) or a real number with values that is accurate to seven significant decimal figures real. , RW) Judgment of the correctness of the learner response
  • cmi.interactions.n.latency (timeinterval (second,10,2), RW) Time elapsed between the time the interaction was made available to the learner for response and the time of the first response
  • cmi.interactions.n.description (localized_string_type (SPM: 250), RW) Brief informative description of the interaction
Jonathan Atleson

Steve, so are you saying that it is the LMS that is responsible for incrementing the interactions._count variable and assigning the unique interaction ID (that Storyline generates) to a new "n" in the interactions array? Or is Storyline supposed to send the "n" in the interactions array?

Our "LMS" is only reporting on interactions.0 and appears to be overwriting that value with each quiz question that comes in, so that only the last question is stored.

On the other hand, SCORM Cloud is storing all of the unique interaction ids that Storyline is sending. I'm just not seeing the interaction.n value in reportage.

Thanks!

This discussion is closed. You can start a new discussion or contact Articulate Support.