My course includes graded questions only. I have chosen to submit all answers at once. If I review the quiz to correct my responses before I submit all answers, the final score I get is unusually low, or even zero, even when all my answers are correct. When I review the quiz at the end, a red bar shows at the bottom saying 'Incorrect', but I know my answers are correct.
Have you looked at your slide properties to see if you're using the "resume saved state" set up? If you're using the "reset to initial" then on a review it will be reset as if the user did not previously answer. Please let us know if you can check into that element and then if you're still seeing the odd behavior especially in regards to your "submit all" set up we'll want to take a look at the .story file.
My apologies - I didn't notice that it was Quizmaker. It does not have the same settings as Storyline - so ignore that from before.
You mentioned users are reviewing the course and also navigating back to previously answered slides? Are they only seeing the 'incorrect' bar when conducting the review? Have they exited the course and resumed? Are you using the latest update of Quizmaker '13, Update 7?
Users see both incorrect bar and correct bar on different questions. When publishing the course, I check the always resume box on restart, so even if users exit the course and resume, they start from the last slide they answered. I am also using the latest update.
I think we'll need to take a look at the .quiz file in terms of how it's set up and then what occurs when testing it. Do you want to share it here with us using the "add attachment" button?
I have attached one of my tests. It isn't the only test that shows a problem. All the tests that are published with the Submit all at once feature show this problem. When the test is published with the Submit one at a time feature, there are no problems.
I forgot to mention that if students choose to submit their answers as soon as they finish the test, without going back to review and correct them, the score they get is correct.
Thanks Iliyan for sharing the .quiz file. Are you seeing this behavior in Flash or HTML5 or both? Also, what LMS are you using to view the content? I see an issue reported to our QA team in regards to the LMS Desire2Learn where the correct answers are shown as incorrect during Review when uploaded to Desire 2 Learn (D2L) LMS and quiz is set to "Submit all at once".
I uploaded your course here to SCORM Cloud so that we could test it out - and if you'd like to send along an answer key that would help me test it better, but also if you can run through it please let me know how it works.
Illiyan's course was also in Quizmaker not Storyline - does your scenario match hers as described above? Are you using the LMS that I mentioned or only viewing the HTML5 output? You mentioned posting another query on this - so we'll keep our eyes out for that thread as well to see if there is a bit more information shared there.
I experience the same problem when I open the quiz.html file. As for quiz_html5, when I get to the last window, the system doesn't respond and I can't submit the answers. The LMS I am using is called Melon, but the problem is present whenever I start a test, through the LMS and when I start it from the hard drive.
I ran through the quiz in the SCORM Cloud and it's all the same.
To make it a bit easier to test, I stripped it down to a first grouping of questions and added the answer right on screen. I followed your steps where I answered some correct and some incorrect, and then went back through before submitting and changed my answers - for some correct and incorrect. When I submitted and saw the final score, it was correct in tallying up my points and the review showed the correct/incorrect bar as expected. Could you test this version out quick and let me know if you see different?
I tried the test, answered all questions correctly, except one, went back to correct only that question. When I submitted, everything was incorrect except this particular question that I retyped.
Sorry - it looks like I linked to the wrong one. Here's the shortened one I was testing around with - you'll see I wrote the answers on the screen. Just to confirm we're navigating through it in the same way, I answer all the questions with at least one incorrect answer, and get to the thank you page. From that page I click on the "return to quiz" pop up button and navigate back through to change my answers. I also tried navigating back the ones I did wrong using the previous button and then navigating forward again using the next button to finally submit the answers. Let me know how this one works for you and what steps you're taking to navigate through, how many you answer incorrectly and then change to be correct, and anything else that you do to replicate the odd behavior.
The page says the course is not currently available. As for the navigation of the course, I don't see any other way to do it. I answer all the questions with a few incorrect answers, and when I get to the final page I click on the 'return to quiz' button. I change my incorrect answers, then navigate forward with the next button and click on the 'submit answers' button. At the end, the only correct answers are the ones that I previously answered incorrectly and then retyped them. The ones that were originally written correctly appear to be incorrect.
I think I figured out what the problem is. I made the box where the student types in the answer very wide. In other tests where that box is regular size, there is no odd behavior. Also, this problem is present only in Fill in the Blank questions. I tested Multiple choice, Word bank, Matching drag and drop and they all work fine.
I'm not sure why my new link didn't work for you - it should allow you to open up the course normally. Since we're having inconsistent results, it may be best to continue troubleshooting this with our Support engineers so that we can take a further look.
You mentioned changing the box size, perhaps that played a role but I'd also want to confirm that you were working as described here, as working off a network or USB drive is known to cause odd behavior as well.
I have a developer who is running into what may be the same issue here. They are have a course built in Studio 13 using Presenter with a Quizmaker quiz at the end. The issue seems to be specific to the multiple response question type and the HTML5 output. I have tested in both Chrome and Firefox with the same results.
With a correct answer submitted by the users the HTML5 version marks it incorrect and the Flash version marks it correct. The issue seems inconsistent though I a see it in questions with 4 of 4 and 3 of 4 answers correct, but a question with 2 of 4 answers correct is working correctly.
I have confirmed they used the latest version, Update 7 and this is the version info from the published presentation.html:
<!-- Created using Articulate Presenter '13 7.7 - http://www.articulate.com --> <!-- version: 7.7.1509.1618 -->
I recreated the issue in a sample course which I have attached both source and output files for. Other than changing it to submit all at once and having the question answers not shuffle everything else is default settings. The quiz in the sample is pretty simple and has the answers as the questions.
Our team is aware of an issue where if you use "submit all at once" in the HTML5 output the answers can be scored incorrectly and they're currently investigating. You mentioned it only happened with two out of your 4 questions though? Were the others set up to be submitted immediately?
For the current issue, I don't have an update or time frame from our team, but the current workaround is to use the "submit individually" to allow them to score while viewing the HTML5 output.
All of the questions in both the sample I attached and in the courses our developer has built have all of the questions in the same submit all at once quiz.
I have 3 Multiple Response questions at the end of the sample, all with 4 possible answers; A, B, C, and D. The first question A, B, C ,and D are the correct answer, the second question A, B, and C are the answer, and the third question just A and D are the answer. The first two are not working correctly, the third one is.
Thanks for sharing more detail about your set up. I tried opening your file but got an error message when trying to do so that it was created in a newer version, but since you created it in Studio '13 and that's what I'm using to open it. Do you have another sample you could share perhaps? And you'll want to make sure you're following along with the general guidelines here.
Hi Ashley, Has a resolution been found to the problem with quizmaker showing incorrect answers even if correct options have been chosen? We are facing this issue with many of our quiz files.
Are you in the same situation that Ashley described above:
Our team is aware of an issue where if you use "submit all at once" in the HTML5 output the answers can be scored incorrectly and they're currently investigating.
If your issue is the same, this is still open with our team and I do not have any further details to share at this time. This thread is linked so we can pop in and update when applicable.
Hi Wim! Do you fall into the 'submit all at once' and viewing the HTML5 content as shared above? If so, I do not have an update to share at this time. This issue is still in the hands of our QA Team.
40 Replies
Hi Iliyan,
Have you looked at your slide properties to see if you're using the "resume saved state" set up? If you're using the "reset to initial" then on a review it will be reset as if the user did not previously answer. Please let us know if you can check into that element and then if you're still seeing the odd behavior especially in regards to your "submit all" set up we'll want to take a look at the .story file.
Hello Ashley,
I am using Articulate Quizmaker '13, and I can't see where the slides properties are located. Is this a feature in Storyline only?
Hi Iliyan,
My apologies - I didn't notice that it was Quizmaker. It does not have the same settings as Storyline - so ignore that from before.
You mentioned users are reviewing the course and also navigating back to previously answered slides? Are they only seeing the 'incorrect' bar when conducting the review? Have they exited the course and resumed? Are you using the latest update of Quizmaker '13, Update 7?
Hi Ashley,
Users see both incorrect bar and correct bar on different questions. When publishing the course, I check the always resume box on restart, so even if users exit the course and resume, they start from the last slide they answered. I am also using the latest update.
Hi Iliyan,
I think we'll need to take a look at the .quiz file in terms of how it's set up and then what occurs when testing it. Do you want to share it here with us using the "add attachment" button?
Hello Ashley,
I have attached one of my tests. It isn't the only test that shows a problem. All the tests that are published with the Submit all at once feature show this problem. When the test is published with the Submit one at a time feature, there are no problems.
Thank you
I forgot to mention that if students choose to submit their answers as soon as they finish the test, without going back to review and correct them, the score they get is correct.
Thanks Iliyan for sharing the .quiz file. Are you seeing this behavior in Flash or HTML5 or both? Also, what LMS are you using to view the content? I see an issue reported to our QA team in regards to the LMS Desire2Learn where the correct answers are shown as incorrect during Review when uploaded to Desire 2 Learn (D2L) LMS and quiz is set to "Submit all at once".
I uploaded your course here to SCORM Cloud so that we could test it out - and if you'd like to send along an answer key that would help me test it better, but also if you can run through it please let me know how it works.
I am having the same issue not in story line but in studio 2013. and am afraid if i can share the file. I have already posted a query on this.
Hi Akhil,
Illiyan's course was also in Quizmaker not Storyline - does your scenario match hers as described above? Are you using the LMS that I mentioned or only viewing the HTML5 output? You mentioned posting another query on this - so we'll keep our eyes out for that thread as well to see if there is a bit more information shared there.
Hello Ashley
I experience the same problem when I open the quiz.html file. As for quiz_html5, when I get to the last window, the system doesn't respond and I can't submit the answers. The LMS I am using is called Melon, but the problem is present whenever I start a test, through the LMS and when I start it from the hard drive.
I ran through the quiz in the SCORM Cloud and it's all the same.
I have attached the answer key of the quiz.
Thanks Illiyan for the answer key and I'm taking a look at the course now using this.
To make it a bit easier to test, I stripped it down to a first grouping of questions and added the answer right on screen. I followed your steps where I answered some correct and some incorrect, and then went back through before submitting and changed my answers - for some correct and incorrect. When I submitted and saw the final score, it was correct in tallying up my points and the review showed the correct/incorrect bar as expected. Could you test this version out quick and let me know if you see different?
I tried the test, answered all questions correctly, except one, went back to correct only that question. When I submitted, everything was incorrect except this particular question that I retyped.
Hi Illiyan,
Sorry - it looks like I linked to the wrong one. Here's the shortened one I was testing around with - you'll see I wrote the answers on the screen. Just to confirm we're navigating through it in the same way, I answer all the questions with at least one incorrect answer, and get to the thank you page. From that page I click on the "return to quiz" pop up button and navigate back through to change my answers. I also tried navigating back the ones I did wrong using the previous button and then navigating forward again using the next button to finally submit the answers. Let me know how this one works for you and what steps you're taking to navigate through, how many you answer incorrectly and then change to be correct, and anything else that you do to replicate the odd behavior.
Hi Ashley,
The page says the course is not currently available. As for the navigation of the course, I don't see any other way to do it. I answer all the questions with a few incorrect answers, and when I get to the final page I click on the 'return to quiz' button. I change my incorrect answers, then navigate forward with the next button and click on the 'submit answers' button. At the end, the only correct answers are the ones that I previously answered incorrectly and then retyped them. The ones that were originally written correctly appear to be incorrect.
I think I figured out what the problem is. I made the box where the student types in the answer very wide. In other tests where that box is regular size, there is no odd behavior. Also, this problem is present only in Fill in the Blank questions. I tested Multiple choice, Word bank, Matching drag and drop and they all work fine.
Hi Iliyan,
I'm not sure why my new link didn't work for you - it should allow you to open up the course normally. Since we're having inconsistent results, it may be best to continue troubleshooting this with our Support engineers so that we can take a further look.
You mentioned changing the box size, perhaps that played a role but I'd also want to confirm that you were working as described here, as working off a network or USB drive is known to cause odd behavior as well.
Hi Ashley,
I have a developer who is running into what may be the same issue here. They are have a course built in Studio 13 using Presenter with a Quizmaker quiz at the end. The issue seems to be specific to the multiple response question type and the HTML5 output. I have tested in both Chrome and Firefox with the same results.
With a correct answer submitted by the users the HTML5 version marks it incorrect and the Flash version marks it correct. The issue seems inconsistent though I a see it in questions with 4 of 4 and 3 of 4 answers correct, but a question with 2 of 4 answers correct is working correctly.
I have confirmed they used the latest version, Update 7 and this is the version info from the published presentation.html:
<!-- Created using Articulate Presenter '13 7.7 - http://www.articulate.com -->
<!-- version: 7.7.1509.1618 -->
I recreated the issue in a sample course which I have attached both source and output files for. Other than changing it to submit all at once and having the question answers not shuffle everything else is default settings. The quiz in the sample is pretty simple and has the answers as the questions.
Hi Ian,
Our team is aware of an issue where if you use "submit all at once" in the HTML5 output the answers can be scored incorrectly and they're currently investigating. You mentioned it only happened with two out of your 4 questions though? Were the others set up to be submitted immediately?
For the current issue, I don't have an update or time frame from our team, but the current workaround is to use the "submit individually" to allow them to score while viewing the HTML5 output.
I also did want to note that the HTML5 output is not supported in Firefox per the documentation here.
All of the questions in both the sample I attached and in the courses our developer has built have all of the questions in the same submit all at once quiz.
I have 3 Multiple Response questions at the end of the sample, all with 4 possible answers; A, B, C, and D. The first question A, B, C ,and D are the correct answer, the second question A, B, and C are the answer, and the third question just A and D are the answer. The first two are not working correctly, the third one is.
Hi Ian,
Thanks for sharing more detail about your set up. I tried opening your file but got an error message when trying to do so that it was created in a newer version, but since you created it in Studio '13 and that's what I'm using to open it. Do you have another sample you could share perhaps? And you'll want to make sure you're following along with the general guidelines here.
Hi Ashley, Has a resolution been found to the problem with quizmaker showing incorrect answers even if correct options have been chosen? We are facing this issue with many of our quiz files.
Thanks!
Gaurav
Hi Gaurav!
Are you in the same situation that Ashley described above:
Our team is aware of an issue where if you use "submit all at once" in the HTML5 output the answers can be scored incorrectly and they're currently investigating.
If your issue is the same, this is still open with our team and I do not have any further details to share at this time. This thread is linked so we can pop in and update when applicable.
Hi Leslie,
I have the same issue, answers are correct but shown incorrect. Is there a solution?
Hi Wim! Do you fall into the 'submit all at once' and viewing the HTML5 content as shared above? If so, I do not have an update to share at this time. This issue is still in the hands of our QA Team.