Help! In a SL2 quiz I recently created, I noticed that students are getting different results for the same responses. In the published quiz, one student answered a question correctly and received feedback indicating that the question was correct. Another student answered the same question correctly and received feedback saying that it was incorrect. This happened for every question in the quiz. Does anyone know why this might be happening? Thanks!
Well that's certainly odd and unexpected behavior! Can you share a bit more about where you're hosting the published output? Is the score being shown incorrectly within the Storyline course or on your web server/LMS? Does the same user always see it as correct and the other as incorrect? Are they all accessing the courses on separate computers/systems or do they share a system? Do you know what web browser they're viewing the courses in?
The published output is being accessed in the same environment. Both students accessed it from our LMS (i.e., they launched it from the same page in our LMS), using the same type of laptop and same browser. In fact, they were accessing it at the same time, during a class. I published the file using the SCORM 2004, 3rd edition setting since the bookmarking feature did not work when publishing in SCORM 1.2. I am pretty sure both students used the Chrome browser.
Thanks for your response! We had a class of 6 students taking the quiz, all at the same time. For the students who did not experience this issue, the course scored perfectly. For the student who received the "incorrect" feedback for every question, the score did not report accurately. She passed the assessment, even though the questions were showing up as incorrect. The quiz was set to an 80% passing rate.
The published output was being accessed in the same environment. All students accessed it from our LMS (i.e., they launched it from the same page in our LMS), using the same type of laptop and same browser. I published the file using the SCORM 2004, 3rd edition setting since the bookmarking feature did not work when publishing in SCORM 1.2. I am pretty sure it was accessed in a Chrome browser.
Thanks for that additional break down of information. For the one user who saw the "incorrect" set up - did the score show up correctly on your results slide and not the LMS, or vice a versa - or were both wrong? Where did it show that she successfully passed the assessment?
For this one user are they able to reenter the course - how does their score show if they try to access it again?
Also, as an FYI responding via email includes your signature here, so you're welcome to edit the post to remove that information.
I scored her assessment using our answer key, and she should have received a score of about 95%. However, the results slide gave her a score of 80%, which was correctly passed to the LMS as 80%. The assessment was telling her that each answer she gave was wrong, so we were expecting to see a much lower score.
By the way, this only happened during the multiple choice portion of the assessment. We also had some simulation exercises in the quiz, which scored just fine. For the multiple choice questions, we randomized the answer choices. Could that have anything to do with it?
This user was onsite several weeks ago for a training session, so we are not able to test whether she can access it again.
If the answer choices being randomized was part of the issue, it would have impacted more than just this user. When you say the assessment was telling her each answer was wrong - do you mean on the "feedback layers" or as a part of the review functionality? If the latter, do you know if the other users looked at using the review as well? Do you know if your slide properties were set to "reset to initial state"? If so, that could show a user that they scored incorrectly as they review the quiz - but since they've already seen the results slide their score would have been passed correctly to the LMS (and in this instance it did pass the score she saw on the results slide). Where are you checking this users' answers with an answer key?
Without a bit more information, I'd guess it's an issue specific to this one user's attempt.
She was getting the "Incorrect" feedback layer after answering the questions. I don't think she looked at the quiz review at all. Also, we had each question set to "resume saved state."
While she was tasking the quiz, I was looking at her answers and comparing it with our answer key, so I know that she was selecting the correct answers. However, she was still seeing the text box on the "Incorrect" layer, as if she had answered incorrectly.
It's certainly bizarre and I'm out of ideas - especially if it was just this one user, and some of the answers tracked/scored correctly based on the 80% scoring. Does your LMS include a report of how the user answered? Did that show the information correctly as well? Have you tried to take the course yourself to see if you could get the same behavior?
I tried the course several times and cannot replicate the issue. Unfortunately, our LMS only tracks the course as passed or failed, so I can't look to see how the user answered. Since it was an isolated case, I am hoping it is just a fluke or computer glitch! Our wifi in the classroom was spotty that day. I wonder of that could have caused a glitch?
11 Replies
Hi, Suzanne -- Thanks for reaching out and welcome to the community!
May I ask if the published output is being accessed in the same environment (as in, an LMS or same web page)?
Please see here for additional details on Publishing and Sharing SL2 Content, and you may also want to check out this System Requirements for Articulate Storyline 2 sheet to confirm that both students are accessing content in a supported format.
Any additional details you can provide would be helpful so we can nail down the source of the issue! :)
Hi Suzanne,
Well that's certainly odd and unexpected behavior! Can you share a bit more about where you're hosting the published output? Is the score being shown incorrectly within the Storyline course or on your web server/LMS? Does the same user always see it as correct and the other as incorrect? Are they all accessing the courses on separate computers/systems or do they share a system? Do you know what web browser they're viewing the courses in?
Thanks for your reply.
The published output is being accessed in the same environment. Both students accessed it from our LMS (i.e., they launched it from the same page in our LMS), using the same type of laptop and same browser. In fact, they were accessing it at the same time, during a class. I published the file using the SCORM 2004, 3rd edition setting since the bookmarking feature did not work when publishing in SCORM 1.2. I am pretty sure both students used the Chrome browser.
Thanks,
Suzanne
Thanks for your response! We had a class of 6 students taking the quiz, all at the same time. For the students who did not experience this issue, the course scored perfectly. For the student who received the "incorrect" feedback for every question, the score did not report accurately. She passed the assessment, even though the questions were showing up as incorrect. The quiz was set to an 80% passing rate.
The published output was being accessed in the same environment. All students accessed it from our LMS (i.e., they launched it from the same page in our LMS), using the same type of laptop and same browser. I published the file using the SCORM 2004, 3rd edition setting since the bookmarking feature did not work when publishing in SCORM 1.2. I am pretty sure it was accessed in a Chrome browser.
Hi Suzanne,
Thanks for that additional break down of information. For the one user who saw the "incorrect" set up - did the score show up correctly on your results slide and not the LMS, or vice a versa - or were both wrong? Where did it show that she successfully passed the assessment?
For this one user are they able to reenter the course - how does their score show if they try to access it again?
Also, as an FYI responding via email includes your signature here, so you're welcome to edit the post to remove that information.
I scored her assessment using our answer key, and she should have received a score of about 95%. However, the results slide gave her a score of 80%, which was correctly passed to the LMS as 80%. The assessment was telling her that each answer she gave was wrong, so we were expecting to see a much lower score.
By the way, this only happened during the multiple choice portion of the assessment. We also had some simulation exercises in the quiz, which scored just fine. For the multiple choice questions, we randomized the answer choices. Could that have anything to do with it?
This user was onsite several weeks ago for a training session, so we are not able to test whether she can access it again.
Thank you again for all of your help!
Hi Suzanne,
If the answer choices being randomized was part of the issue, it would have impacted more than just this user. When you say the assessment was telling her each answer was wrong - do you mean on the "feedback layers" or as a part of the review functionality? If the latter, do you know if the other users looked at using the review as well? Do you know if your slide properties were set to "reset to initial state"? If so, that could show a user that they scored incorrectly as they review the quiz - but since they've already seen the results slide their score would have been passed correctly to the LMS (and in this instance it did pass the score she saw on the results slide). Where are you checking this users' answers with an answer key?
Without a bit more information, I'd guess it's an issue specific to this one user's attempt.
She was getting the "Incorrect" feedback layer after answering the questions. I don't think she looked at the quiz review at all. Also, we had each question set to "resume saved state."
While she was tasking the quiz, I was looking at her answers and comparing it with our answer key, so I know that she was selecting the correct answers. However, she was still seeing the text box on the "Incorrect" layer, as if she had answered incorrectly.
Hi Suzanne,
It's certainly bizarre and I'm out of ideas - especially if it was just this one user, and some of the answers tracked/scored correctly based on the 80% scoring. Does your LMS include a report of how the user answered? Did that show the information correctly as well? Have you tried to take the course yourself to see if you could get the same behavior?
Hi Ashley,
I tried the course several times and cannot replicate the issue. Unfortunately, our LMS only tracks the course as passed or failed, so I can't look to see how the user answered. Since it was an isolated case, I am hoping it is just a fluke or computer glitch! Our wifi in the classroom was spotty that day. I wonder of that could have caused a glitch?
I hope so too! I haven't heard of a wifi outage causing something like that - but I guess anything is possible?
Not too reassuring - but fingers crossed it doesn't happen again!
This discussion is closed. You can start a new discussion or contact Articulate Support.