An Assessment Journey #3: The Measure of All Things

I've posted previously about the development of a Confidence Based Assessment that enables  an assessment to report not just on whether a learner is correct or incorrect but also on how confident they are that their answer is accurate.  

This means that a scoring algorithm is able to group individuals into various categories including:

• Confidently Knowledgeable - a correct response and they were very confident that they were correct.
• High Understanding with low confidence - a correct response but low confidence.
• Misunderstanding with High Confidence - the response was incorrect but the learner was very confident it was correct

(see: https://community.articulate.com/discussions/articulate-storyline/an-assessment-journey-2-confidence-based-assessment-in-storyline-360)

But how to report and and track all of this assessment data? 

SCORM doesn't provide sufficient data points to enable an assessment with this number of data points to be tracked. 

Ultimately our solution will track all this data by using an xAPI compliant LMS and an LRS, but in the interim I needed a solution that would enable me to track and report on over 125 data points for each assessment session. 

This included not just session information such as date and time and user identification but scores for "correctness" and "confidence" across almost 140 questions within six topics.

The solution I used was to export the data with a "phone home" script that exported all the assessment data into a Google Sheet. 

This method is explained in great detail by Bastiaan Timmer in this post: https://community.articulate.com/discussions/articulate-storyline/articulate-storyline-export-to-google-drive 

The images below give you some idea of the metrics which this approach has enabled me to track and report on (the black squares are covering redacted information):

High Level Dashboard showing number of attempts and passes, total users on platform and daily activity for the assessment.

High Level Dashboard showing what % of learners get each question correct on  their first attempt.

Individual Scores for Each Unit including Confidence Scores

Will this technique work for everyone? 

Probably not - but it's worth considering if you have a great deal of data and if you can make it work for your situation.

This example shows what I believe is one of the great benefits of using Storyline 360 - the ability to include Java Scripts which can significantly extend what is possible!