Confidence-based assessment in Quizmaker
Jan 02, 2012
Has anyone tried to implement confidence-based assessment, or certainty-based marking in Quizmaker.Confidence-based assessments evaluates the learner for correctness as well as their confidence with respect to the answer.As multiple choice questionsand true/false questions can be guessed it is hard to assess if the learner has guessed the correct answer or if they really know the material being tested. With confidence based assessment you ask how confident the learner was that the answer was the correct answer.
I'm currently trying to implement this in regard to testing knowledge of Microsoft Office products e.g Word Excel.
Any help would be appreciated.
Ger
17 Replies
I'm not sure how to implement certainty-based marking assessments in Quizmaker, but it may indeed be possible.
In the past, we have developed these assessments in SCORM-compliant elearning using custom programming in Flash. And because there was no place to record the additional certainty-based data in the LMS, we created a companion reporting website for admins to use, which showed details such as: how learners did as a group on each lesson's assessment and how a learner did on each quiz question.
I wish I could be more helpful with implementing it in Quizmaker. If you're interested, we did put together an article on our experiences with certainty-based marking that you might find interesting. ( go to http://www.docntrain.com/resource_view.php?id=43 )
Hi Ger,
I'm working on a project for LINGOS where we're doing something similar. It's more around a "readiness plan" where the user certifies their level of preparedness. The activity was inspired by this Thanksgiving Readiness survey.
Now, we're building this in Storyline, but, you could use Jeanette's technique for creating multiple-response questions in Quizmaker. Or, another great option is Jeanette's assess & categorize learners demo in Quizmaker.
Please let us know if you think those ideas could work... We'd be happy to help you work it out.
Try playing different quiz games. that will give u some ideas.
Thanks to all of you for your replies I was away from the computer for a few days.I've had a quick look at Jeanette's multiple-response questions in Quizmaker, I think i may be able to implement using this.
Thanks again for all your help Ger
Hi all,
Was the confidence based assessment functionality built into Storyline? If so, can you point me to some instructional material for it? We are hoping to assess confidence in diagnosis of different medical conditions and hopefully bring up a summary at the end along the lines of "Out 19 questions of which you answered confidently (90%+) , 17 were correct, etc.
Has Quizmaker or Story line added any more confidence-based capabilities? I really like the multiple response tutorial, but I don't think it will work when it comes to confidence, because the user confidence level is not a right/wrong question.
Hi Julee,
This thread is a bit older, so I'm not sure if anyone else is still subscribed but there hasn't been a change to any of the question types recently. Although I'm not sure what you mean by confidence based questions? Could you share a bit more information and then perhaps folks in the community will be able to weigh in on ideas for how to accomplish this in Storyline or Quizmaker's current question format.
We've got clients who already do confidence based assessments.
In their existing quiz, they'll ask a question--and after the answer is selected AND while on the same slide--the confidence assessment comes up to ask the learners level of confidence in the answer they chose.
One particular client uses this to flag individual users who got the question wrong with a high confidence.
We're trying to replicate this with Quizmaker, but so far I can only put the quiz question on one slide followed by a Likert question on the next. Separating the question and the confidence assessment is unacceptable to the client. We need to make this work using Quizmaker. Do you have any ideas on how we can make this work?
We are also considering Storyline 2 to implement these quizzes. Is there a good way to ask a question and ask for the confidence in the answer on the same slide? Thanks!
Hi Julee,
I reached out to a few of our team who know a bit more about instructional design and they weren't entirely clear on what you meant by confidence based questions either. Storyline and Quizmaker only allow for one question per slide, so the set up you described wouldn't be possible with the built in functionality of the questions.
Given that we can only have one question per slide, let's consider a way to do this scenario. The user is asked a multiple choice question and submits the question. In Storyline 2 is there a way to branch based on the answer so we can show the content of the question and the selected answer on the next slide when we ask the user their confidence in their answer?
This client later uses the combination of a wrong answer with high confidence on the report to follow up with users who are not understanding health and safety content.
Hi Julee,
Branching and feedback by choice is available for certain question types, including multiple choice.
This is a really old thread - but just in case someone is ever curious...it is possible to create a confidence based assessment within Storyline 360 using variables and various triggers; it's not possible to do so using the out of box Quiz and Assessment tools provided in the package.
Here is a working example:
https://360.articulate.com/review/content/ae859529-92c3-48ee-976f-43caa951e206/review
Kevin this is a great example, I imagine you could even add some xapi statements to capture the results.
It’s amazing that years later this is still so relevant - If only articulate could add this type of quiz as standard it could be real point of difference.
Hi Kevin,
This is a great example. I think I have some ideas of how it was executed, but would you mind sharing how you built this?
Brian
Hello Brian!
Since this is a dated discussion, Kevin may no longer be subscribed. Instead, I'd recommend using the Contact Me button in their profile to get a hold of them directly!
Very cool build. Thanks for sharing!