LMS reporting "undefined" or blanks for some questions

Hello!

We have created a math assessment for our LMS (Desire2Learn), and upon testing it in on the site I can see that the reporting is getting some strange results. For some of the multiple choice questions, it is coming back as "undefined" in both (and sometimes just one) of the correct response and learner response fields. Also, there are issues with the the fill-in-the-blank questions, with some being blank for both correct and learner response, and some having just a learner response (blank for correct). I've included a screen shot of the report. As far as I can tell, all of the questions are following the same format for how they've been created and set to report. I'm not sure why some questions are reporting fine, while others are having these issues. Any help would be greatly appreciated!

Thanks,
Wanda

84 Replies
Dave Cox

Hi Debra,

I have on occasion seen this behavior. The solution that I've found works best is to delete the question slide and re-create it from scratch. When you enter the text for your questions, be sure to enter them in the Form view from the question. 

I've also noticed that if you copy and paste your question text from word or powerpoint, you will at times get some odd hidden text that seems to cause this behavior. I always copy my text into notepad first. This strips out the hidden characters. Then I copy and paste the text into Storyline.

I also noticed that you are using SCORM 1.2. I have to use that too, as our version of SCORM doesn't seem to work well with any other reporting method. A problem you may have noticed with SCORM 1.2 in Storyline is that it doesn't report the question text. I have written a patch to fix that if you want it. Let me know if you do, and I'll post it with instructions on how to enable it.

Emma Herbert

So i've just gone and deleted all of the spaces from the questions in a different module. They are no longer reporting as 'undefined' but for some questions I am only getting the first letter of an answer reported unfortunately.  

For example where I have a list of answers all beginning with the same letter, I am unable to see exactly which answer option a learner has selected. I am just getting a report of "d,d,d,d,d" where the answers are things such as 'data can be gathered...' 'data can be imported...' 'data can be referenced externally...'.

Has anybody else encountered this before? 

Robert L

In some ways I'm glad to see others who struggle with this. As mentioned earlier, we've eliminated most of our undefined errors by removing special characters (", ', $, #, and others). I've also asked my team to make sure they use the Notepad method.

But, I still see cases were a question response comes over as "undefined" for a few users but comes over correctly for others. I don't know if that's tied to user behavior (e.g. using multiple browser windows during a session), network issues, a learner going back through the questions and changing an answer before clicking Submit, or just random.

It's very strange and frustrating.

Emma Herbert

Hi Robert, i've eliminated the 'undefined' problem but i'm now having an issue with only the first letter of a lot of my multiple response questions being reported. I first thought it was all multiple response questions, but I have had some multiple response questions record the full response in the Interactions Report. I'm now wondering whether it might be related to word length which is causing the issue

Robert L

I wanted to provide a little more data to show you what we're seeing in the reporting. I hope this comes through (I made a number of the field generic):

Lesson Name

Catalog Course Code (ID)

Interaction ID

Question

Question Type

Correct Response

Learner Response

Result

Score

Time Spent in course

First-Last Name

Interaction Date

Exam Name

Exam LMS Code ID

Scene1

Mike Walls
Robert Loverin

(...) I still see cases were a question response comes over as "undefined" for a few users but comes over correctly for others. (...)

Hi Robert,

Does any of your questions reveal the correct answer to the learner after they've answered it incorrectly?

I've found that this can cause Undefined learner response to be logged. Here's some specific steps to reproduce the problem (I have a hard to explaining it another way hehe):

  1. First thing I did was set the slide to "Reset To Initial State" when it is revisited.
  2. I create a couple variables for the slide, "Attempts" (A number) and "IsFinished" (A boolean).
  3. When the user answers the question incorrectly for the 2nd time, "IsFinished" is set to true and the "Continue" button on the "Incorrect" layer jumps to the same slide (Causing the slide to be reset to initial state).
  4. When the timeline of the slide starts, a trigger checks if "IsFinished" is true and if it is, the submit button is hidden and the correct answer is shown to the learner.

In testing, I found that because of step 4, a 3rd response is logged in the LMS as "undefined" as (I believe) Storyline submits the interaction response. A quick solution/way to confirm this as an issue was to change the "Incorrect" layer's "Continue" button to simply jump to the next slide.

Hopefully this makes sense. Good luck :)

Robert L

That does make sense and it’s something we can check. I believe the exam that had the undefined error was a Submit All exam that allows the learner to revisit the questions afterwards (but does not tell them the correct answer). The learner can go back through all of the questions before submitting, so perhaps something in the way the slide remembers a change could trigger confusion in the question?

I appreciate the steps below.

Elena Urbanova

Hi,

I would like to ask you about the status of this issue. We have also experienced it in Storyline 360 and two diffrent LMS - Totara and NetDimensions.

We found that when we use language specific characters, for example: ž,š,ť,á,é... then the answers report to LMS as undefined - undefined is the correct answer and the student response when he chooses the correct answer. If he chooses the incorrect answer, then correct answer is undefined, but the student choice is just fine.

And the other thing is - the first question works ok, no matter what.

I hope this will help to resolve the issue if it is not already. We cannot not to use special characters - they are part of our language (Slovak) and simply cannot be replaced or avoided.

Thank you

Ashley Terwilliger

Hi Elena,

I see that my colleague Vevette responded to your Support case as well and let you know that this is an issue we've reported to our team. Once we have any additional information we'll be certain to share here and as a part of your Support case. Thanks for letting us know that you've been impacted by this as well and I'm really sorry if it's slowing you down. 

Ali Goulet

Hey Everyone,  

I have some great news! 

We just released another update for Storyline 360, and included a few important fixes and new features that you'll see in the release notes here 

This update fixes the issue you reported where non-English characters and symbols would cause quiz data to be undefined in LMS logs and reports.

Just launch the Articulate 360 desktop app on your computer and click the Update button for each application. Details here.

Please let us know if you have any questions, either here or by reaching out to our Support Engineers directly!