6 Replies
Christine Hendrickson

Hi Jim,

I'm afraid that I'm not quite clear on what you're wanting to do here. Can you share a little more detail, please?

Are you using Storyline or Studio for your projects? 

Are you wanting to display multiple attempts and scores, etc. in Articulate Online, in a results slide, etc.?

In Articulate Online you should be able to run reports that will show detailed information for multiple attempts at a quiz. Maybe this is what you're looking to do?

Thanks!

Richard Giraud

Hi Christine

I work with Jim and I think I can elaborate on the issue.

We need to track multiple attempts for the following use cases.  (Cases 1 and 2 are the ones that we are concerned about, as case 3 works.)

Case 1 - Single Session, Single Pass

Assume the question has infinite retries:

  1. The user fills out the question incorrectly.
  2. The user clicks the 'Submit' button.
  3. Storyline tells the user that their answer is wrong with the 'Try Again' layer/dialog.
  4. The user dismisses the dialog.
  5. The user modifies their answer, correcting it.
  6. The user clicks the 'Submit' button.
  7. The user is told that their answer is correct with the 'Correct' layer/dialog.
  8. The user navigates to the corresponding Results page.

Is it possible to retrieve the score for the question as generated in both steps 2 and 6?  I.e., is it possible to see both in a report or to export them?  My experience is that it's only possible to see the result from step 2; the result from step 6 is discarded.

Case 2 - Single Session, Multiple Passes

Assume the question allows no retries.

  1. The user fills out the question incorrectly.
  2. The user clicks the 'Submit' button.
  3. Storyline tells the user that their answer is wrong with the 'Incorrect' layer/dialog.
  4. The user dismisses the dialog.
  5. The user navigates to the corresponding Results page.
  6. The user clicks a button which resets the test, allowing them to take it again.
  7. The user returns to the question.
  8. The user fills out the question correctly this time.
  9. The user clicks the 'Submit' button.
  10. The user is told that their answer is correct with the 'Correct' layer/dialog.
  11. The user navigates to the corresponding Results page.

Is it possible to retrieve the score for the question as generated in both steps 2 and 9?  I.e., is it possible to see both in a report or export them?  My experience is that it's only possible to see the result from step 2; the result from step 9 is discarded.

Case 3 - Multiple Sessions

Assume the question allows no retries.

  1. The user fills out the question incorrectly.
  2. The user clicks the 'Submit' button.
  3. Storyline tells the user that their answer is wrong with the 'Incorrect' layer/dialog.
  4. The user dismisses the dialog.
  5. The user navigates to the corresponding Results page.
  6. The closes the test.
  7. The user launches the test again and choose to not continue.
  8. The user returns to the question.
  9. The user fills out the question correctly this time.
  10. The user clicks the 'Submit' button.
  11. The user is told that their answer is correct with the 'Correct' layer/dialog.
  12. The user navigates to the corresponding Results page.

In this case, it appears that AO tracks the results from both of steps 2 and 10 as separate passes, which works fine for our purposes.

The SCORM module in Moodle is capable of tracking and presenting the results of all 3 cases through the Interactions report.  Can AO track and present them, too?

Thanks,

Richard

Christine Hendrickson

Hi Richard,

Thanks so much for the information! I appreciate the clarification.

For Case 1:

Is it possible to retrieve the score for the question as generated in both steps 2 and 6?  I.e., is it possible to see both in a report or to export them?  My experience is that it's only possible to see the result from step 2; the result from step 6 is discarded.

I'm afraid not. You're completely correct, the initial attempt and score is not stored. The score is "passed on" when the actual question completes and the learner advances to the Results Slide. The information is then submitted when the learner continues on or exits from the Results Slide (you've probably seen the triggers designated to this slide).This won't pass the original attempt to the LMS or the results/score displayed to the user. 

For Case 2:

Is it possible to retrieve the score for the question as generated in both steps 2 and 9?  I.e., is it possible to see both in a report or export them?  My experience is that it's only possible to see the result from step 2; the result from step 9 is discarded.

Correct, this is the same as Case 1 in that the initial attempt isn't stored or reported, because it hasn't passed on from the course yet. It's still active when they land on the Results Page, but once they click the "Retry Quiz" button, that score for the quiz or question is reset. 

The SCORM module in Moodle is capable of tracking and presenting the results of all 3 cases through the Interactions report.  Can AO track and present them, too?

Not with a single course or SCO honestly. However, if you're wanting to do this with AO, you could try this unsupported method. However, that truly wouldn't accomplish the same thing you're wanting to do, since you're wanting to see multiple scores for different attempts on the same question(s).

What you could do is break it up so that the learner is taken to a "fresh" version, or copy, of the initial question and have that tied to a "collective" results slide. I'm still not sure if that would work for you, and to be honest it would probably be a bit tedious, depending on how many questions you're working with.

Out of curiosity, why are you wanting to see the failed or initial attempt along with the passing/correct attempt? I'm sure there's good reasoning behind it, just haven't seen this type of question and curious about the goal behind it, especially if learners are going to be allowed to attempt and pass the quiz again in the future by revisiting.

Thanks Richard! Hope you're having a great day :)

Christine

Richard Giraud

Out of curiosity, why are you wanting to see the failed or initial attempt along with the passing/correct attempt? I'm sure there's good reasoning behind it, just haven't seen this type of question and curious about the goal behind it, especially if learners are going to be allowed to attempt and pass the quiz again in the future by revisiting.

One of the advantages of being able to see multiple results is that it becomes possible to track the progress of a student on a question when multiple attempts are allowed.  Did the student get it on the first try? If not, was it a minor error or a major one?  Did they deduce the answer or did they use brute force?

I understand that it's a bit too complex for AO's built-in reporting but it would be nice if the information was available through CSV and XML export and, ideally, through the AO API.

Richard Giraud

Hi Christine

We are using HTML5 publishing with update 2 (update 3 has some bugs that prevent our using it). When using custom Activity Reports, the behaviour seems to differ from what you've described.

  1. Information is submitted immediately.  That is, as soon as the user clicks the 'Submit' button, the results are recorded; there is no need to proceed to the results slide.
  2. The first non-incomplete submission is stored, not the last.  This was tested with a Pick One with different scores for each option and unlimited tries.  However, when tallying up the score, the last value is used.

It's possible this behaviour has changed in update 3 but it would probably be good to kick it over to QA to confirm that item 2 has been corrected; it doesn't seem like the expected behaviour and may cause some confusion.

Richard

edited for correctness