Confidence-based assessment in Quizmaker

Has anyone tried to implement  confidence-based assessment, or certainty-based marking in Quizmaker.Confidence-based assessments evaluates the learner for correctness as well as their confidence with respect to the answer.As multiple choice questionsand true/false questions can be guessed it is hard to assess if the learner has guessed the correct answer or if they really know the material being tested. With confidence based assessment you ask how confident the learner was that the answer was the correct answer.

I'm currently trying to implement this in regard to testing knowledge of Microsoft Office products e.g Word Excel.

Any help would be appreciated.

Ger

14 Replies
Patti Quinn

I'm not sure how to implement certainty-based marking assessments in Quizmaker, but it may indeed be possible.

In the past, we have developed these assessments in SCORM-compliant elearning using custom programming in Flash. And because there was no place to record the additional certainty-based data in the LMS, we created a companion reporting website for admins to use, which showed details such as: how learners did as a group on each lesson's assessment and how a learner did on each quiz question.

I wish I could be more helpful with implementing it in Quizmaker. If you're interested, we did put together an article on our experiences with certainty-based marking that you might find interesting.  ( go to http://www.docntrain.com/resource_view.php?id=43 )

David Anderson

Hi Ger,

I'm working on a project for LINGOS where we're doing something similar. It's more around a "readiness plan" where the user certifies their level of preparedness. The activity was inspired by this Thanksgiving Readiness survey.

Now, we're building this in Storyline, but, you could use Jeanette's technique for creating multiple-response questions in Quizmaker. Or, another great option is Jeanette's assess & categorize learners demo in Quizmaker.

Please let us know if you think those ideas could work... We'd be happy to help you work it out.

Marcel Mihulka

Hi all,


Was the confidence based assessment functionality built into Storyline? If so, can you point me to some instructional material for it? We are hoping to assess confidence in diagnosis of different medical conditions and hopefully bring up a summary at the end along the lines of "Out 19 questions of which you answered confidently (90%+) , 17 were correct, etc.

Ashley Terwilliger

Hi Julee, 

This thread is a bit older, so I'm not sure if anyone else is still subscribed but there hasn't been a change to any of the question types recently. Although I'm not sure what you mean by confidence based questions? Could you share a bit more information and then perhaps folks in the community will be able to weigh in on ideas for how to accomplish this in Storyline or Quizmaker's current question format.

Julee Jaeger

We've got clients who already do confidence based assessments.

In their existing quiz, they'll ask a question--and after the answer is selected AND while on the same slide--the confidence assessment comes up to ask the learners level of confidence in the answer they chose.

One particular client uses this to flag individual users who got the question wrong with a high confidence. 

We're trying to replicate this with Quizmaker, but so far I can only put the quiz question on one slide followed by a Likert question on the next. Separating the question and the confidence assessment is unacceptable to the client. We need to make this work using Quizmaker. Do you have any ideas on how we can make this work?

Ashley Terwilliger

Hi Julee,

I reached out to a few of our team who know a bit more about instructional design and they weren't entirely clear on what you meant by confidence based questions either. Storyline and Quizmaker only allow for one question per slide, so the set up you described wouldn't be possible with the built in functionality of the questions. 

Julee Jaeger

Given that we can only have one question per slide, let's consider a way to do this scenario.  The user is asked a multiple choice question and submits the question. In Storyline 2 is there a way to branch based on the answer so we can show the content of the question and the selected answer on the next slide when we ask the user their confidence in their answer?

This client later uses the combination of a wrong answer with high confidence on the report to follow up with users who are not understanding health and safety content.

Kevin Hayes

This is a really old thread - but just in case someone is ever curious...it is possible to create a confidence based assessment within Storyline 360 using variables and various triggers; it's not possible to do so using the out of box Quiz and Assessment tools provided in the package.

Here is a working example:

https://360.articulate.com/review/content/ae859529-92c3-48ee-976f-43caa951e206/review