LMS reporting "undefined" or blanks for some questions

Hello!

We have created a math assessment for our LMS (Desire2Learn), and upon testing it in on the site I can see that the reporting is getting some strange results. For some of the multiple choice questions, it is coming back as "undefined" in both (and sometimes just one) of the correct response and learner response fields. Also, there are issues with the the fill-in-the-blank questions, with some being blank for both correct and learner response, and some having just a learner response (blank for correct). I've included a screen shot of the report. As far as I can tell, all of the questions are following the same format for how they've been created and set to report. I'm not sure why some questions are reporting fine, while others are having these issues. Any help would be greatly appreciated!

Thanks,
Wanda

84 Replies
Ashley Terwilliger

Hi Wanda,

Thanks for sharing the image here. Based on that image, it looks like for the fill in the blank your LMS is only showcasing those that include numbers? I suspect the other ones are all text based. As for the multiple choice, I'm unsure based on this alone, but I'd suggest reviewing the information here on troubleshooting common LMS reporting and tracking issues. You may also want to test your course at SCORM Cloud which is an industry standard for testing SCORM content, and see how the course behaves there and what type of information you're able to gather.

Please let us know how it behaves at SCORM Cloud, and if you're able to see the necessary information there you'll want to connect with your LMS team about why certain questions may not be reporting as expected.

Wanda Blackett

Hi Ashley,

I set up an account on SCORM Cloud and did a quick test. Unfortunately, I am still seeing the same kind of results with blanks (' ') for some answers, and strangely with many of the multiple choice I am now getting one character results - some are a number and some are a letter. These multiple choice responses should definitely be longer than one character. You can see in my screen shot some of the responses (I did a very quick test and answered with 11 for most text questions).

Ashley Terwilliger

Hi Wanda,

Can we also get a look at your .story file so that we can see these particular questions and how you've published? I suspect you're publishing for SCORM 1.2 - which does report less information about the answers themselves, but shouldn't provide "blank" answers - which I see two of in this recent interaction data you shared.

Wanda Blackett

Tried to add my file - but it's too big for here. It's about 42MB.

I am using SCORM 1.2 - however I've tried just about every other publishing settings I could think of and get pretty much the same results with the undefined and blank answers.

For the posting I did recently, that one had much more than 2 blank answers. It is only for the fill in the blank questions that I am getting some empty fields - sometimes for the correct response, sometimes for learner response, and sometimes for both. But I do know that every fill in the blank should have a correct response in the field (they are all text), and I responded to every question, so there should be no blank fields. You can even see on some that are blank that I have a "correct" response, but we can't see anything there.

It is the Multiple choice questions that I am getting the "undefined" result in the correct and learner response fields. There are only 2 of them in the example I posted - but it varies for each test as I am drawing from several question banks during the assessment. All of the questions were put together the same way - so I don't know why some are working and some are not.

This assessment was a huge project for us to put together, and I am frustrated that we may not be able to use it as it was intended. Certianly we can see if learners answered correctly or not - however we need to see learner's responses in case we need to correct a mark (fill in the blank doesn't always get the right "correct" answer as intended"). Also, as it is an initial assessment for learners coming into our literacy program, we very much need to have a record of how they did.

Our next major project is to do an English assessment in the same format as our math assessment. The English assessment is going to rely even more heavily on the fill in the blank questions to be marked by hand, as beyond a word or two, we can't predict correct answers when typing sentences.

Ashley Terwilliger

Hi Wanda,

Yes, my apologies I was referring to the two multiple choice questions. If your story is too large to upload here in the forums, you can share it with us here.

I understand your frustration, and without taking a look at the file it's hard to determine if the issue is within the Storyline file itself, the LMS or how it's being reported through SCORM 1.2, as SCORM 1.2 does report less overall information than say SCORM 2004.

Please let me know once you've been able to send along the file and we'll take a look.

Wanda Blackett

Hi Ashely,

I had started a support ticket at that link a few hours before your last post. I did make note of this forum conversation though, and have since uploaded my storyline file to that case. It is Case #00413325 if you want/are able to get involved on that side of things.

Thanks,

Wanda

Juanjo Haro

Hi... I have a similar issue.

We are testing a course on a Saba LMS that sens us our client to test the packages and we check the log we can see "undefined" in 2 of the 3 questions of the quiz (interactions). First we have 2 questions and got 1 undefined, we added 1 more question and now we see 2 undefined. I'm using update 6

I will star a ticket to send the file



cmi.suspend_data = 2t5g80b09040506070a0YX1001411P010110111101211013110111111011101111110112111001411j12131311101010100000~2Q30~2L36002010~261cb101001a1a123nE0L2402402407A200_10240kD2300000118_default024024000010~2b1cb101001a1a103bE0Q240240240kD23000O0118_default07A200A107A20000024024000010~2b1cb101021010123NZ0Q2402402407A200o207A200000kD22000c1118_default02402400001010

cmi.interactions.0.id = Scene1_QuestionDraw41_Slide2_MultiChoice_0_0

cmi.interactions.0.type = choice

cmi.interactions.0.student_response = Aquellos_obligados_a_tributar_en_Estados_Unidos_por_motivos_de_residencia_o_ciudadan_a.

cmi.interactions.0.correct_responses.0.pattern = Aquellos_obligados_a_tributar_en_Estados_Unidos_por_motivos_de_residencia_o_ciudadan_a.

cmi.interactions.0.result = correct

cmi.interactions.0.weighting = 1

cmi.interactions.0.objectives.0.id = fatca_prueba_v2_escala

cmi.interactions.0.time = 14:51:23

cmi.core.score.raw = 66.66

cmi.core.score.max = 100

cmi.core.score.min = 0

cmi.interactions.1.id = Scene1_QuestionDraw41_Slide3_MultiChoice_0_0

cmi.interactions.1.type = choice

cmi.interactions.1.student_response = undefined

cmi.interactions.1.correct_responses.0.pattern = undefined

cmi.interactions.1.result = correct

cmi.interactions.1.weighting = 1

cmi.interactions.1.objectives.0.id = fatca_prueba_v2_escala

cmi.interactions.1.time = 14:51:25

cmi.interactions.2.id = Scene1_QuestionDraw41_Slide1_MultiChoice_0_0

cmi.interactions.2.type = choice

cmi.interactions.2.student_response = undefined

cmi.interactions.2.correct_responses.0.pattern = undefined

cmi.interactions.2.result = wrong

cmi.interactions.2.weighting = 1

cmi.interactions.2.objectives.0.id = fatca_prueba_v2_escala

cmi.interactions.2.time = 14:51:29

cmi.core.lesson_status = passed

cmi.core.session_time = 0000:00:35.91

Denise Simao

I am using SumTotal and am also getting "undefined" on just only 1 of 4 multi-select questions responses in SCORM 1.2 published file. I have also seen it in multiple choice in another module (SCORM 2004) that I just assumed was a SumTotal issue so I opened a ticket with them. I will also create a ticket with Articulate.

Denise Simao

I think I found the problem! There were spaces in the beginning of the responses for these questions. I put them there when the responses didn't line up, no matter what I did and it didn't look right so I tried to manually align the responses. I took the spaces out and voila - it works and shows responses instead of null (scorm 2004) or undefined (scorm 1.2). Now, I did have this happen to another module that does not have spaces and is not consistent in sending the "undefined" over to the LMS so I will keep the ticket open for that one.

Wanda Blackett

Alright - I'm getting (more) frustrated. I'm hedging my bets, and going for both the direct support with a ticket (am going back and forth with them), as well as posting here. Someone must be able to figure this out!

Just so I don't have to retype everything, this is a copy/paste from my latest e-mail to support:

Starting from scratch, I have remade the entire project. I’ve published as a SCORM 1.2 (the 2004 version is just not good for us as it changes the appearance of the webpage on our LMS for some reason, so I must use 1.2). I am attaching our SCORM results as a word file. Please have a look at the comments I’ve added in the last column for the questions that were missing the Correct and/or Learner Responses. There is an obvious pattern there as the problems only occur when dealing with certain number formats. Every time I use a dollar amount or decimal (with or without the $ symbol), the information isn’t sent through. The same thing happens for time – for all time values (ex. 7:30), the fields are left blank. If the result is a whole number only, then it is transmitted just fine, and included in the report.

At this point I can only conclude that there is an underlying issue within the code that goes through when it comes to numbers that include decimals, colons, negative numbers and fractions. It is bizarre to say the least, however it is something that we absolutely need resolved! Our program is a Literacy and Basic Skills program, and part of our online training and assessments are focused on numeracy. Having our hands tied with regards to what number formats are acceptable for assessment purposes is not going to work for us.

Ashley Terwilliger

Hi Wanda,

I see that you emailed Cleo about this issue, and you'll be able to share a copy of your .story file with him as well. I do see an issue that was reported to our QA team in regards to the $ causing issues with the interaction data being reported as undefined, so I shared that with Cleo as well so that he could look into how it behaves for you with the decimal and colon as well.

Please feel free to keep us posted here and I'll continue following along in your case.

Debra Bumgarner

I am just noticing this same problem with some of our eLearning tests...learners are answering all of the questions but the response column on the LMS may show "undefined"...even for correct answers, so we know that a choice was made! I have not attempted to re-publish using the version 6 update, but will do that in a few minutes and test the theory. Ashley, it seems like you are saying that is what fixed this problem? Please advise if there is anything else that may be causing this to happen.

Ashley Terwilliger

Hi Debra,

This thread has a few different issues in it, so without knowing your exact set up and what's happening I don't want to say for certain that Update 6 will fix it - but it's where I'd start. Next, you'll also want to review the troubleshooting steps in this article and look at testing this course at SCORM Cloud which is an industry standard for testing SCORM Content. If the behavior persists there we'll want to take a look at your course. 

Debra Bumgarner

Hi Ashley,

Thank you for responding so quickly. I did re-publish one of the tests with this issue using my Storyline version, which is the most recent update (6). The same questions are showing a response of "undefined" on our LMS progress details for answers that were scored as correct. Unfortunately, due to company restrictions I am unable to test this on the SCORM cloud. I have asked my manager if we can send this file to you guys to look at, and I am awaiting her answer right now.