User Completion issue on LMS (Docebo)
Dec 01, 2023
Hey there team,
We've encountered an error on our end. Some users on our LMS (Docebo), are able to receive completion of courses even though they have not yet finished the course. The way our course is structured, it throws a completion to the user reaches the final page of the course.
This page is locked behind the results slide, which only directs a user to that completion page after passing the assessment with a score of 80%. There is no way for a user to navigate to this completion page without doing so. This page is removed from the course flow, and only accessed with a button interaction on the successful assessment results slide. A majority of users are receiving completion in this manner. (as intended) There are maybe a dozen users out of hundreds that managed to bypass this, and received a completion but have no recorded score of having taken the final assessment.
I've taken a look at some user's data, and it seems like some users haven't taken the final assessment, or the LMS isn't recording that they did. I've come to this conclusion because the LMS isn't recording their final assessment results. I could be wrong. It's not consistent though, some are only recording part of the assessment data, and others are not recording any
I haven't been able to replicate this problem on my end on SCORM cloud. Does this sound like a familiar issue to anyone? It could also be possible that the LMS is doing something that is affecting completion. I'm not sure where to even start with this issue without any way to replicate.
3 Replies
Hi Tony, the links provided don't appear to be working. Is it possible for you to provide these again?
Hi Tony, further on this:
"There are maybe a dozen users out of hundreds that managed to bypass this, and received a completion but have no recorded score of having taken the final assessment."
This could indicate that the LMS is incorrectly setting a completion status of completed under unusual circumstances, for example the content/system crashes, of the way the user exits the content or it times-out.
It's unfortunate, but I have seen some behaviour like this with LMS before, and it can be very difficult to determine the root cause of the issue, or even implement code to fix it.
It sounds to me like your course is set-up correctly in terms of tracking.
The only other thing I would check is "mastery_score". If there is a mastery_score entry in the imsmanifest.xml document, it can sometimes cause unexpected completion of modules. When a mastery_score is defined in the manifest file, this is an indication to the LMS that it should determine completion (passed/failed) based on the mastery_score and any score sent to the LMS. If a score is sent to the LMS, and a mastery_score has been set in the manifest, the LMS will then automatically evaluate the user score against the master_score to determine pass/fail. I don't think this is the case in your situation, but worth a mention.
Do you have enough data to determine the percent of users with unusual completion statuses? If it is under 0.5% of users (5 in every 1000), you are likely seeing something that will be very difficult to replicate and may not be fixable.
You could also look at the users and see if their is any commonality with their location or how they are accessing the LMS. This may give clues such as an unstable network, poor bandwidth etc.
Some things to consider anyway.
Thanks for the direction. The manifest file does not indicate that we are using mastery_score, so i don't think that is the issue.
In terms of location, they seem to be all over the place. But I'm currently looking into LMS side related issues, like getting timed out, disconnected, etc. that could have triggered the issue.