Issues with Completion reporting in Cornerstone when using xAPI: Failed assessments showing as "completed" in the user's transcrtipt.

Mar 20, 2023

Hello everyone!

                            I'm hoping you can help here. My team and I are exporting our first xAPI courses to publish in Cornerstone (CSOD) and while they play correctly, the completion of the courses with assessments are not tracked as we would expect. 

What we would basically like is for a course to only be marked as completed in Cornerstone if they successfully pass the assessment. However, while looking at the xAPI statements the exported storyline file sends to the LMS/LRS, we noticed that "failed" statement for the assessment looks as follows (text redacted in bold, italics, and underlined for security purposes): 

{
  "actor": {
    "objectType": "Agent",
    "name": "Username",
    "account": {
      "homePage": "OurLMSHomepage.com",
      "name": "000001"
    }
  },
  "verb": {
    "id": "http://adlnet.gov/expapi/verbs/failed",
    "display": {
      "en-US": "failed"
    }
  },
  "object": {
    "objectType": "Activity",
    "id": "urn:articulate:storyline:ourcourse",
    "definition": {
      "type": "http://adlnet.gov/expapi/activities/objective",
      "name": {
        "und": "ourcourse"
      },
      "description": {
        "und": "course description."
      }
    }
  },
  "result": {
    "completion": true,
    "success": false,
    "duration": "PT1M28.51S",
    "score": {
      "scaled": 0.2
    }
  },
  "context": {
    "registration": "123443563",
    "contextActivities": {
      "parent": [
        {
          "objectType": "Activity",
          "id": "urn:articulate:storyline:ourcourse"
        }
      ],
      "grouping": [
        {
          "objectType": "Activity",
          "id": "urn:articulate:storyline:ourcourse"
        }
      ]
    }
  },
  "timestamp": "2023-03-20T20:37:42.508Z",
  "id": "123098",
  "authority": {
    "objectType": "Agent",
    "name": "username",
    "account": {
      "homePage": "OurLMSHomepage.com",
      "name": "00001"
    }
  },
  "stored": "2023-03-20T20:37:48.76"
}

The piece I'm interested in is the "Result" statement:

 "result": {
    "completion": true,
    "success": false,
    "duration": "PT1M28.51S",
    "score": {
      "scaled": 0.2
    }

While is marks success as "false", it marks completion as "true", and it seems that this makes Cornerstone automatically assume that the course can be moved to completion status in the user's transcript and, in turn, issue a certificate of completion. Their Support Center article on the matter states: 

"The CSOD LRS (xAPI Statement Viewer) will record valid verbs as they are sent from the course, however, to mark the course complete on the Transcript and move the course to the Completed Tab - the following conditions need to occur:
The course will send any of the following verbs:

  • passed
  • completed
  • failed
  • attempted 

and the result tag in the statement has the value of completion = True"

Source: https://csod.my.site.com/supportcentral/s/article/xAPI-When-does-a-course-move-to-Completed-Tab-on-Transcript?language=en_US

Based on this CSOD article, I can understand why even if the user fails the assessment, they are still marked as complete in the transcript. However, I believe this is a big problem because reporting-wise, the user show show as incomplete if a test fails. While I'm looking on the CSOD end on possibilities to address this, I would like to ask for this community's support in figuring out if there is a way to either:

  1. Create a send xAPI custom statement that informs that if the course is failed (our success is false), then to set the value of completion to "false".
  2. If this is not possible via Articulate statements, does anyone know if the Storyline-based xAPI statement that sends this information can be adjusted via the export files?

Any support will be appreciated! 

13 Replies
John Morgan

Hi David,

Good news! I have some information that will hopefully get you going on the right path. Here is an article about creating custom xAPI statements in Storyline 360.

Also, have you tested your course in SCORM cloud to see if it reports the same thing?  It's an online LMS environment that can be used to figure out whether the issue is with the LMS or the Story file. If you test your course on SCORM cloud and it reports the way you would like, then your issue is LMS based.

Thanks for reaching out!

Kelly Auner

Hi Shannon,

Thanks for reaching out and I'm sorry to hear you're experiencing this issue!

  • It looks like Rachel was using Rise 360. Are you seeing this issue with Rise 360 or Storyline?
  • Is this happening with all courses or a specific one? 
  • Have you tried testing the course in  SCORM Cloud?

If you're comfortable sharing your course with our team, we'd be happy to take a closer look. You can get in touch with our support team here!

Rachel Lally

We discovered a workaround by adding a storyline block that shows only after an assessment is completed (passed). When the user clicks on the storyline block it closes the course and the assessment result is recorded correctly as completed (the block won't appear if they have failed). This meant we had to update all of our courses with assessments to include the storyline block but saved us from exporting them as Scorm 2004 and losing our data/having two versions of the same course on Cornerstone. If anyone else has a fix or workaround that was effective I would love to learn about it.

Torgeir Lien Wennersberg

Thank you for the response, Rachel! Does that mean you do not get the test result reported unless they pass on the test? I was thinking of putting a completed trigger on the success layer, however I cannot report the result on the failure layer as this would give the users a completed status in Cornerstone. Did you find any workaround enabling you to report the result on failed attempts?

Rachel Lally

No, we can't distinguish between 'in progress (failed)' and 'in progress'. But all we need to know is whether the learner has passed the assessment and therefore 'completed' the training. We can go into users transcripts and data in Cornerstone if we want to know specifics but usually, we don't require that level of reporting. As I said, it is a workaround - so not ideal but it serves our needs right now and might be helpful to you. (Otherwise, we were facing the horror of re-exporting and managing two versions of every course and compiling reporting etc. (i.e. a nightmare!).

Torgeir Lien Wennersberg

This is really useful to us, thank you. I think we can manage with only knowing in progress or completed. After all that is what we have been reporting on with SCORM. I guess we would have to look to Cornerstone Support in order to get this implemented properly, however my experience with that is it takes forever.