Forum Discussion
Issues with Completion reporting in Cornerstone when using xAPI: Failed assessments showing as "completed" in the user's transcrtipt.
Hello everyone!
I'm hoping you can help here. My team and I are exporting our first xAPI courses to publish in Cornerstone (CSOD) and while they play correctly, the completion of the courses with assessments are not tracked as we would expect.
What we would basically like is for a course to only be marked as completed in Cornerstone if they successfully pass the assessment. However, while looking at the xAPI statements the exported storyline file sends to the LMS/LRS, we noticed that "failed" statement for the assessment looks as follows (text redacted in bold, italics, and underlined for security purposes):
{ "actor": { "objectType": "Agent", "name": "Username", "account": { "homePage": "OurLMSHomepage.com", "name": "000001" } }, "verb": { "id": "http://adlnet.gov/expapi/verbs/failed", "display": { "en-US": "failed" } }, "object": { "objectType": "Activity", "id": "urn:articulate:storyline:ourcourse", "definition": { "type": "http://adlnet.gov/expapi/activities/objective", "name": { "und": "ourcourse" }, "description": { "und": "course description." } } }, "result": { "completion": true, "success": false, "duration": "PT1M28.51S", "score": { "scaled": 0.2 } }, "context": { "registration": "123443563", "contextActivities": { "parent": [ { "objectType": "Activity", "id": "urn:articulate:storyline:ourcourse" } ], "grouping": [ { "objectType": "Activity", "id": "urn:articulate:storyline:ourcourse" } ] } }, "timestamp": "2023-03-20T20:37:42.508Z", "id": "123098", "authority": { "objectType": "Agent", "name": "username", "account": { "homePage": "OurLMSHomepage.com", "name": "00001" } }, "stored": "2023-03-20T20:37:48.76" }
The piece I'm interested in is the "Result" statement:
"result": { "completion": true, "success": false, "duration": "PT1M28.51S", "score": { "scaled": 0.2 }
While is marks success as "false", it marks completion as "true", and it seems that this makes Cornerstone automatically assume that the course can be moved to completion status in the user's transcript and, in turn, issue a certificate of completion. Their Support Center article on the matter states:
"The CSOD LRS (xAPI Statement Viewer) will record valid verbs as they are sent from the course, however, to mark the course complete on the Transcript and move the course to the Completed Tab - the following conditions need to occur:
The course will send any of the following verbs:
- passed
- completed
- failed
- attempted
and the result tag in the statement has the value of completion = True"
Based on this CSOD article, I can understand why even if the user fails the assessment, they are still marked as complete in the transcript. However, I believe this is a big problem because reporting-wise, the user show show as incomplete if a test fails. While I'm looking on the CSOD end on possibilities to address this, I would like to ask for this community's support in figuring out if there is a way to either:
- Create a send xAPI custom statement that informs that if the course is failed (our success is false), then to set the value of completion to "false".
- If this is not possible via Articulate statements, does anyone know if the Storyline-based xAPI statement that sends this information can be adjusted via the export files?
Any support will be appreciated!
- JohnMorgan-c50cFormer Staff
Hi David,
Good news! I have some information that will hopefully get you going on the right path. Here is an article about creating custom xAPI statements in Storyline 360.
Also, have you tested your course in SCORM cloud to see if it reports the same thing? It's an online LMS environment that can be used to figure out whether the issue is with the LMS or the Story file. If you test your course on SCORM cloud and it reports the way you would like, then your issue is LMS based.
Thanks for reaching out!
- PhilMayorSuper Hero
I mistakenly thought you could control this by changing the reporting and tracking to incomplete, but not an option for xAPI.
I have not tried this, have you tried moving the trigger for the results slide that submits the results to the success layer? That way it will only submit on pass (not ideal).
- RachelLally-e19Community Member
Did you ever find a resolution to this, I am experiencing the same with courses exported with xAPI in Rise to Cornerstone.
Hi Rachel, and thanks for reaching out!
I'm sorry to hear that you've hit this snag, and I see you've connected with my teammate, Darrell, in a support case. You’re in excellent hands! We’ll continue the conversation there.
- ShannonOKeefe-bCommunity Member
Same issue for us with Cornerstone
Hi Shannon,
Thanks for reaching out and I'm sorry to hear you're experiencing this issue!
- It looks like Rachel was using Rise 360. Are you seeing this issue with Rise 360 or Storyline?
- Is this happening with all courses or a specific one?
- Have you tried testing the course in SCORM Cloud?
If you're comfortable sharing your course with our team, we'd be happy to take a closer look. You can get in touch with our support team here!
- ShannonOKeefe-bCommunity Member
It’s happening in storyline. So far it’s just for one course, we need to do more testing. I was able to test in scorm cloud, and it worked.
Shannon O’Keefe
Great to know that it is working in SCORM Cloud, Shannon! Since things are working as expected in SCORM Cloud, I suggest contacting Cornerstone LMS, as this issue could be isolated to your specific LMS settings.
Please let us know your findings, Shannon! Happy to help you further!
- TorgeirLienWennCommunity Member
Hi. Did anyone resolve this issue? We are experiencing the same defect exporting xAPI from Storyline 360 to Cornerstone. Any help would be appreciated!
- RachelLally-e19Community Member
We discovered a workaround by adding a storyline block that shows only after an assessment is completed (passed). When the user clicks on the storyline block it closes the course and the assessment result is recorded correctly as completed (the block won't appear if they have failed). This meant we had to update all of our courses with assessments to include the storyline block but saved us from exporting them as Scorm 2004 and losing our data/having two versions of the same course on Cornerstone. If anyone else has a fix or workaround that was effective I would love to learn about it.
- TorgeirLienWennCommunity Member
Thank you for the response, Rachel! Does that mean you do not get the test result reported unless they pass on the test? I was thinking of putting a completed trigger on the success layer, however I cannot report the result on the failure layer as this would give the users a completed status in Cornerstone. Did you find any workaround enabling you to report the result on failed attempts?
- RachelLally-e19Community Member
No, we can't distinguish between 'in progress (failed)' and 'in progress'. But all we need to know is whether the learner has passed the assessment and therefore 'completed' the training. We can go into users transcripts and data in Cornerstone if we want to know specifics but usually, we don't require that level of reporting. As I said, it is a workaround - so not ideal but it serves our needs right now and might be helpful to you. (Otherwise, we were facing the horror of re-exporting and managing two versions of every course and compiling reporting etc. (i.e. a nightmare!).