Exams Marking Questions Improperly

Hi Phillip

what slide revisit setting do you have on the quiz slides?

When reviewing incorrect questions, the settings are set to 'Review only incorrect questions.'

Is that what you are asking?

No Phillip what are the slide properties on the quiz slide bottom right hand corner when they revisit - reset to initial - auto decide - resume saved state?

Thanks for clarifying.

The slide properties are all set to "When revisiting: Automatically decide."

Just to make sure we are on the same page...what happens is they submit the exam (questions are set to submit all at once), then they have the option on the results slide (via buttons) to Review Results. Some of the questions are marking as incorrect even though they answered them correctly (we have confirmed at least 6 questions so far).

Thanks again for your help.

Happy to take a look if you want to share the .story file

Thank you, Wendy! I appreciate the offer. They are very large files, so I submitted one of the exams as a case (case number 02133469) for you to take a look.

I look forward to hearing what you might discover.

Thanks again!

Hi Philip

I'm just a regular Jo on the forum not Articulate staff so I won't get to see the file - interested to know the outcome if you wouldn't mind posting when you find out.  Have a great day!

Ok...I see. No worries.

Thank you for being my Hero as much as possible!

I'll post on here if they are able to figure anything out.

Thanks for contacting our support team as well, Philip. I can see that you are currently working with Anna :)

We are still trying to find a solution.

In the meantime, I was able to find some interesting information regarding one the examples.

One question during the post exam review that came up as Incorrect does not appear in the SCORM session report at all. Thus it appears that question was not presented to the user nor did it affect the score. What might cause a question not presented during an exam to show up during  a results review?

All ideas welcome Thanks in advance!

So I did some more digging. In the SCORM session report for the questions being brought up, those are showing as having no user response. However, in the review, the screen shot shows the questions as answered. What makes this more concerning is that when I tested this on my end and skipped questions, they show up in the review as unanswered.

What might cause questions to:
1. show up as incorrect in the review even when they show as answered correctly?
2. show up as having no user response in the SCORM session report even though they show as answered in the review?

Thanks in advance!

Hi Philip,

I've had the same issue with correctly answered questions being marked as 'incorrect' (incorrect banner at bottom of screen) during review. All question slides are marked as "When revisiting: Automatically decide."

At first this seemed to be related to a large number of folks taking the test at the same time (~60) but then issues were experienced at lower volumes (~30) and then even a group of three individuals quality checking a course.

The common elements so far appear to be the use of questions banks in all cases and outputting as SCORM 1.2 in order to accommodate the SCORM requirements for our LMS.

Browser (Chrome) and version are almost universally consistent in our organization.

Reproducing the issue is very hit or miss. Not even clear if the same questions pose the issue each time. 

All questions have been reviewed to ensure they are setup correctly and work correctly in preview mode and once published.

Is this a Question Bank issue? Perhaps Question Banks and SCORM 1.2 don't play well together?

If anyone's experiencing the same issue then please let me know.

If anyone has the solution then definitely let me know :)

Thanks!

Hi, Neil.

One possible cause for incorrect feedback during review is that the suspend_data element within your SCORM 1.2 course is exceeding the limits of your LMS.  This would come into play only when a learner has suspended and resumed the course, which might explain the difficulty you've had with consistently reproducing the problem.

To confirm this root cause, you can enable LMS debug mode to watch your suspend data flow back and forth between your content and your LMS. By doing so, you can verify that the data returned by your LMS upon resume is the same data that your content sent upon suspend.

Please connect with our team if you have any further questions on this--we'd be happy to help!

Thank you for the response Justin.

I don't anticipate that anyone suspended and resumed the course (timed quiz course) but I'll try the LMS debug mode option. 

Thanks again.

P.S. Your article on Debugging made this a breeze!