Correct answers in exam are marked incorrect when student exits exam and returns.

Oct 12, 2023

Jason Swan
By Jason Swan

Hello, we are stumped by this issue. We have an Exam lesson that draws from 8 question banks. We are finding that when a student exits the Exam and then returns to the Exam from the LMS, whether prior to submit or after they submit, all questions they answered prior to their exit are marked incorrect in review. For example, if the student leaves the Exam at question 3, then launches the lesson again from the LMS and completes the Exam, questions 1 and 2 will be marked incorrect, regardless of how they answered the question. Here's a screen shot showing the student's answer, the checkmark indicating the correct answer (they match) and the Incorrect feedback.

Incorrect screenshot

If the student exits the Exam after they Submit but prior to reviewing the results and then returns to the Exam, all questions in the Exam will be marked Incorrect.

My first thought was that maybe the cmi.suspend_data variable was not passing data correctly or parsing the data correctly when the student re-launches the Exam. However, the students return to their previous location correctly, like to the question they exited from. Also, all of their previous question responses are correctly marked when they return. So, I don't think the issue is related to suspend_data.

Additional information:

  • We are using Cornerstone LMS.
  • We are using Storyline 360 version 3.78
  • The Exam is designed with Next and Previous buttons for navigation through the questions. After the last question page, the student is advanced to a Submit page where they are given the opportunity to review their answers or submit the Exam for grading. After they submit the Exam, an instructor must enter a password to reveal the student's score and gives them a button to review the questions with their answers and the correct answers marked. It is this review, after the instructor unlocks the results, that shows the Incorrect feedback.
  • The Exam consists of 25 questions with a few questions drawn randomly from each of the question banks.
  • The "On restart:" Player Resume property is set to "Always resume"
  • The "When revisiting:" property for all questions in all question banks are set to "Resume saved state".
  • The Exam reports Pass/Fail to the LMS based on the Incorrect information. 

Question

I have searched for a solution for several hours over the past couple of days and most solutions seem to be to set the "When revisiting:" property to "Resume saved state". That is not the issue here.

I should mention that previous versions of this exam, which is updated at least once a year, have not displayed this aberrant behavior. Our recent edits to this project were limited to modifying some of the questions in one or two question banks. A few questions were deleted from the banks and a few questions were modified. 

I would really appreciate any suggestions y'all could make to help us with this! BTW, I can share the .story file, but I'll need to share it privately rather than in this public discussion.

Thank you!

Jason

 

9 Replies
Jose Tansengco
7 months ago10/13/23 at 1:26 pm (UTC)

Hi Jason, 

Thanks for sharing a detailed description of the behavior you're experiencing and for listing all of the troubleshooting steps that you've already taken. This definitely helps us streamline the process of identifying what's causing the behavior on your end. 

I went ahead and opened a support case on your behalf so we can get you in touch with our support engineers who can help test the issue with you. Your case is in good hands, and someone from our team will be reaching out shortly via email!

Jason Swan
7 months ago10/13/23 at 6:59 pm (UTC)
Jürgen Schoenemeyer

Hi Jürgen,

That does seem like the same problem. Unfortunately, if there was a solution, it isn't posted in the discussion thread. 

Jason Swan
6 months ago10/19/23 at 11:24 pm (UTC)

@Jose Tansengco After receiving an initial recommendation from a Customer Support Engineer, I replied back that the solution he proposed did not fix the problem. I have since reached out to him again. It has been 5 days and I haven't heard back from him. Is there a way for me to look up my case online (I have the case number) so that I can get additional troubleshooting support? I'm feeling like I was just dropped since the first solution didn't work.

Jason Swan
6 months ago10/19/23 at 11:33 pm (UTC)

For anyone who might have this issue in the future, I did get a recommendation from an Articulate Customer Support Engineer. After reviewing my .story file, this was his recommendation:

If you are using the built-in Graded, Survey, and Freeform questions in Storyline 360, each slide needs to have a "Submit interaction" trigger. This trigger validates the question and saves the answer before moving on. Apart from this "Submit interaction" trigger, you must also have a "Submit results" trigger for the scores to be submitted to the LMS. Both these triggers will save the information to the LMS. If we don't have a "Submit interaction" trigger, the question will be reported as incorrect.

I went through and added the Submit interaction trigger to each question in our question banks. The Submit results trigger was already applied to the "Submit" button. I had high hopes for this solution since the logic made sense. Unfortunately, after updating the exam, we are still consistently seeing correct answers marked as incorrect (the student's response shows in review and the checkmark indicating the correct response also shows, and they often match, but the feedback shows "Incorrect") if a student leaves the exam before viewing results. This is common as the wifi in our facility isn't too spiffy and our students frequently experience dropped connections, which causes the page to time out and close, requiring the student to launch the exam again. They are returned to the last question they answered, or the submit page if they've already answered the questions, so they don't see a problem until they review the results.

I'll post more when I am able to connect with customer support again.

Sign In to Reply