Questions answered correctly, but marked as incorrect

Hi, using Articulate Storyline update 8 with HTML5 output viewed in Safari on the iPad. I have had a few complaints from some users reporting they answered a question correctly in the quiz, but it was marked incorrect. I have 1 or 2 screenshots to show this.

Has anyone come across this before?

Most of the reports have been on one particular 'freeform drag and drop' question, but I've also had complaints about a 'fill in the blank' and 'multiple choice'.

The question bank is set up with next/prev buttons, with the final question having the submit button.

I've tested the course content myself a few times and never had a problem, and it's a minority who have reported this, but the users have sent screenshots to show the problem.

Thanks!

52 Replies
Judy Nollet

Look carefully at the screenshots to check whether the learners really did answer correctly -- or just think they did.

  • I've had people misinterpret the review slides. They think the marks inside the boxes are the correct answers (when those are actually their answers) and the marks next to the boxes are their answers (when those actually indicate the correct answer). Then they retake the quiz and give the same wrong answers, see the same review slide, and think something is wrong with the program. In such cases, I just explain how to interpret the review slide.
  • I once also had someone who understood the review slide, but she insisted her answer was correct. In that case, I sent screenshots from the course where the content matched what the program said was the correct answer.

I hope your issue is as easy to solve. 

FYI:  The situations described above were multiple choice questions. Fill-in-the-blank can be tricky, because an extra space or a misspelling can prevent an otherwise correct answer from matching. For those, you might have to set the program to accept multiple entries, i.e., minor variations of the correct answer. I haven't had any issues with drag-and-drops, so I can't think of any possible errors (aside from misinterpreting the review).

Good luck!

Daniel Halliwell

Hi, thanks for your suggestions. They have definitely answered correct as I compared their screenshot to my STORY file, and having just run through the quiz a few times I have just seen it for myself. It's a 'fill in the blank' with the answer being '1500' or '1,500', i know I typed it correctly but also the user sent me two of her screenshots as she attempted the quiz twice and it was incorrect both times. For me it was incorrect on my 6th attempt of the quiz.

(in this process, I have also discovered when you 'retry' the quiz it does not remove the previous answer from the text box on the 'fill in the blank' question, which is a different matter. All other types of question are 'blanked' except 'fill in the blank')

As for the drag and drop, there's 4 boxes and 4 images which lock into place once dropped over a box. They're either in the box, or back in the starting position. All 4 have been placed in the correct order but users are sending me screenshots with the review marking it 'incorrect'.

I remember a previous version of Storyline had a problem with submitting all questions at once at the end of the quiz in HTML5, it would mark every question incorrect, and we had to use 'submit' on each slide. Could this have possibly returned intermittently?!

I will remove company branding and import the two questions into a new project so I can share it on here (and easier to test, it's a random 20 from 40 questions so it takes a while to get the question to appear!)

Thanks

 

Emily Ruby

H Daniel!

If you want the text to be removed from the fill in the blank, you will need to set the slide properties to "reset to initial state" upon revisiting. This will clear the text out on a retry of the quiz.

I tested a file from my end using 1500 and 1,500 and was able to get a correct response 10 times in a row.

I will be on the lookout for your file when you can attach it.

Daniel Halliwell

Hi, I've not had chance to create the new file but I have had another report from a user today on a different quiz.

I use prev/next buttons so the user can scroll through the quiz. The last question accrues zero points, and simply asks them to confirm they have answered every question and want to submit their answers. There is only 1 answer to select, and the submit button won't submit the results unless this question is answered.

The user has sent me a screenshot of this slide in the review being shown as 'incorrect'. It's impossible to be incorrect, because there is only 1 possible answer. What is strange is the review slide does not show this 1 answer as being selected; the circle is solid white as opposed to having a grey circle inside it to show it has been selected. But you can't submit the question without it being answered, so how can the review show this as 'not selected' and therefore show as incorrect?!

I hope that makes sense, I can't work out why it would do this especially as it is intermittent.

I'll try get some files uploaded tonight or tomorrow.

Thanks

Daniel Halliwell

Hi, I made a mistake in my post as I am actually still on update 7! I'm hoping to have update 8 installed in the next 2 weeks. I still have on going issues with questions being marked incorrect, and also questions in the question bank not appearing on the slides for the user. This also causes them to fail, because they can't get 100% if the question is missing. Again, intermittent. Close the course and start from scratch and it will work ok. Here's some screenshots to hopefully show examples of questions being marked incorrect. I'm sure I'm not setting anything up wrong in the question because it also marks most users correct!

oren savidor

Hi, 

I'm experiencing the exact same problem as Daniel described above - matching drag & drop question (freeform) is answered correctly while in the Review Quiz mode it shows the red incorrect banner. This behavior can be replicated each time and across different computers, this is not even 5HTML related issue as I'm viewing it on my desktop.

Any solution to this yet?

Thanks,

Oren.

 

Jennifer Johnston

I have a similar problem, which I reported to support but have not heard anything back about. In my case, the reportage from SCORM Cloud directly shows the learner answer matching the correct answer but it is still marked as "incorrect."  It is hard to tell exactly what is going on because the multiple choice answers seem to be coded (for example, the answer is shown as "s,s"). There are both random question bank questions and regular quiz questions. The results are very inconsistent, sometimes they are graded correctly and sometimes they are not. The slides are set to "Automatically decide" and the users aren't allowed to review the questions.  Any suggestions? Thank you.

Ashley Terwilliger-Pollard

Hi Jennifer,

You'll only see the number as a part of the email confirmation you receive, and it'll come from the address of Support@articulate.com and the From name should be Justin Grenier our Director of Customer Support. If you don't see that, please check your junk/spam folder to see if it went there. I tried looking based on your name as listed here alone, but it's a fairly common one so wasn't able to narrow it down. 

Jennifer Johnston

So I am looking further at the report from SCORM Cloud and it looks like the results are displayed as the first letter of the choice option. As an example, there might be 4 options, all start with the latter "s."  The correct response shows as "s,s" and the user has selected "s,s" but they might have selected different "s" options.  So, it is possible the answer is correct or incorrect but I can't tell from the report. So maybe it isn't a problem, or maybe it is. Is there some other way to get SCORM to report on the interactions, the selection vs. the correct response?

Ashley Terwilliger-Pollard

Hi Jennifer, 

It's hard to find a case for you based on your name here alone - could you share your email? Did you email Support@articulate.com directly vs. using the form? 

As for the report and information share, it's dependent on the types of reports you're looking at - but here is the data Storyline will send to your LMS or SCORM Cloud.