Problems when exiting and re-entering course on a question (commit?)

We are experiencing a serious problem with exiting and re-entering exercises in courses created with Storyline 3, presumably linked to one of the latest updates.

This is what used to happen up to a couple of weeks ago:

To our knowledge and practical experience, slides created with Storyline 3 always only did a commit when you changed to the next slide, resulting in information not being kept when I exited the course on one slide and re-entered on the same slide. For this reason, if a user answered a pick one question ending on the “Correct” or “Incorrect” layer and closed and reopened the course without going to the following slide first, both the variables set in the question and the question score itself were just not registered. So if for example they answered correctly, closed, and then re-entered and answered correctly, only the first result would be kept and only 1 point would be scored.

The question would be reset completely even if on the “Correct” layer I set a variable that said that the user would have to automatically open that layer if that variable was already true: this is something we usually do in order to prevent the user from answering a question twice (if they go back to the question they just see their original feedback layer), but it never worked when hard closing the course, supposedly because the commit hasn’t happened yet.

We have programmed our courses around this behaviour for months now and even if we weren’t happy about the chance for the user to answer the questions again after seeing the feedback (potentially it means that they can retake each question until they make it right), we’ve made it work with no major bugs.

 

It seems to us that this behaviour has changed lately.

Now, the course seems to commit without changing slides, resulting in everything that has been set (variables and question score) to be kept even if we hard close the course on the exercise and re-enter.

So, the two following cases happen:

• If the question simply resets, without blocking the user from answering, the question can be counted twice. Example: the user answers correctly, closes the slide when on the “Correct” layer, re-opens the slide, answers again, the question gets counted 2 points instead of 1.
• If we use the method described above to send the user directly on the layer corresponding to the original feedback, it does, so the variables are already registered. But it does something strange to the score: if for example the user answers correctly, then closes, the question is counted correct in the temporary score. Then if I re-enter, it’s like the original positive score is erased (maybe the question is being submitted again but without selecting an answer so it’s registered as wrong?). So in the end, with the course programmed this way, if I answer 10 question but I exit and re-enter after answering each one, I get a final score of 10% (the problem described seems to happen to questions 3 to 10 and not on questions 1 and 2, although I wouldn’t rely much on temporary scores before the results slide submits them; the final score is unequivocally wrong though. The questions are programmed exactly the same). If the 10 questions are answered correctly without ever exiting the course, it’s 100%.

 

Obviously this is not ok either way as the possibility to close and reopen the course cannot be denied to users, and this behaviour would make the quiz results meaningless, or block users when in reality they reached the passing score. We would love to keep the new behaviour (so the user can’t answer the same question twice) as long as it’s working correctly with quiz scores.

We have tried different solutions linked to triggers but nothing works: in the end it looks like as soon as the timeline of the question is restarted, some “submit interaction” action that we cannot see happens and it registers the question as wrong erasing the previous result.

 

Some final useful notes:

• We’ve always used the setting “Reset to initial state” for the slide, mainly because the “Resume saved state” was not working at all when exiting and re-entering (resulting in the user being blocked unless we reset the course manually).
• We export in SCORM 2004, 3^rd edition
• We run our tests both in ScormCloud and in the target LMS (in this case Docebo) with the exact same behaviour.

 

Has anyone experienced the same problem and was able to solve it?