Question Bank review behaviour oddities

Hello Sarah! If I remember correctly, the issue you were encountering with the question bank was related to using the Review layer on your quiz questions in a customized way...i.e., making certain object(s) appear or not appear during the quiz review, based on whether the learner had gotten the question right or wrong. While that technique works great with regular questions, youi're right that there is an issue with the behavior when the questions are part of a question bank. This has been logged with our QA and Dev teams, but right now we don't have an estimate on when it might be addressed. 

For now, if you plan to use the quiz review in this way, I'd recommend using regular questions (not question banks) for your quizzes. Or, you're certainly welcome to attach your story file here with a detailed description of the behavior you want to create, and we can see if we can come up with some alternative methods of accomplishing what you have in mind. 

Hi Jeanette

Thanks for that. That's fine, just checking I hadn't missed anything. Ok, I'll have a ponder what I'm really trying to achieve and get back to you if I need anything.

Hi Jeanette,

I am also using a question bank for randomizing the questions and when using a review button with these question banks and because it is random, the review button seems to think all of the questions are wrong and it seems to be waiting for you to select the correct answer because the review does not remember if you have answered the question and if it was correct.

Do you know if they are going to fix bug in the future? I guess for now we will just include the retry quiz button and omit the review quiz button.

Hi Wendy,

Are the answers actually selected when you're reviewing the questions? Or does it show up with no answer selected and an "Incorrect" response? 

If no answer is selected, it may be due to a revisiting state. Are you able to share your .STORY file here, or send it over to us privately?

Hello Christine

It shows up with no answer selected and an "Incorrect" response? 

Hi Wendy,

Thanks for getting back to me! It sounds like it may be an issue with revisiting those slides. What are your revisiting settings for the question slides? If you haven't already, try setting the slides to "Resume saved state". 

If that doesn't work, are you able to share the file with us? You can post it to this thread by using the "paperclip" icon in your response. Alternatively, you can submit the file in a support case. 

You'll be able to attach the .STORY file on the second page of the following form:

Articulate Support - Submit a Support Case

Please be sure to include a description of your issue. Please also include the URL for this thread in the form. Also, please share the case number with me. You're welcome to do so here, or in a private message. This way I can follow the progress of the case and update this thread.

Thanks!

Hey everyone

[quote]I am also using a question bank for randomizing the questions and when using a review button with these question banks and because it is random, the review button seems to think all of the questions are wrong[/quote]

I also encounter the above. While with Wendy the answers given get lost, in my case the correct answers are remembered, but the bar indicating a correctly or incorrectly answered question is red and states incorrect for each and every correctly answered question.

I was initially setting the module up with all questions in succession for testing purposes, there it worked. Now that the contents are deemed presentable, I moved the slides to several corresponding question banks and draw from those banks for the final module. Now the aforementioned behaviour occurs.

I did a couple tests on our own LMS but also on SCORM Cloud. The behaviour is consistent on both platforms.

First run of the module, I end the module with a score of 100% (I knew the answers :p ):

Without closing the module hit the Review Quiz button and the review states the correctly solved question:

Close the module, launch it again, I get varying results back, normally 0% (0 points), in this specific run for some reason unknown to me 10 points and 4%, instead of the originally reached 100%:

I can now go back using the navigation buttons, using the Review Quiz buttons or the Menu/Table of Contents, the outcome is the same, the question is rated incorrect although previously reviewed as correct:

I guess this has to be turned in a case submission...?

Hi Joël,

Just out of curiosity, what do you have for the revisiting settings on the question slides? Do you have them set to "Resume saved state"? If not, that may be one thing that's causing the display of the "incorrect" bar when the answer was scored as "correct". 

Might be worth checking, if you haven't already :)

If that doesn't help, yes, it may be best to let us take a look. You're welcome to post the .STORY here and I can take a look, or you can send it over to our support team. 

Thanks!

Christine

Hi Christine

For most slides it's set to auto, for specific slides it's set to resume the saved state. I also already submitted the .story file to you guys.

I did some further testing and found out the following: the traffic when relaunching the course is correct, our LMS sends the correct stuff as can be seen when carefully looking at the browser console output:

[quote]

data[cmi.core.lesson_status] => passed

Hi Joël,

Thanks very much for getting back to me. I was able to find the case you submitted (#00329383 for my reference). I see that the case has already been escalated and Jonathan is working on the case. 

If you're able to find any additional information after you run to those tests with the static slides, please let Jonathan know by responding to the confirmation email that was sent out for the case. Or, if you'd like, you're welcome to post the information here and I can pass it along to Jonathan.

I'll be sure to keep an eye on the case and follow up with you if there are any updates.

Thanks again!

Christine

Me again. I'm really fighting things here

My original module was built using a couple of scenes. Most scenes had 4-5 slides, of which most were "random" slides but also some fixed slides. Meaning a scene was built with for example 3 "pull-1-random-question-from-bank" slides and a regular, static question slide. Mixing "static" and random slides seems to break Storyline's internal evaluation as well as state and data restoration logics. I changed my module now as follows:

1. I moved all static slides to a respective question bank
2. I removed all slides from a scene except one question-bank question
3. I set that question to pull in 4 slides from the respective question bank
4. I set the previously static question to be always included in those slides pulled from the bank

This (mostly) solved the issue of the score being totally off upon a second launch, the result slide now reports the score which is also stored in the LMS. It also (mostly) solved the issue of correctly solved slides bearing the red "Incorrect" banner, correctly solved slides now bear the green "Correct" slides where applicable.

It does not solve the visited state not being remembered in the menu/table of contents.

It also still fails at evaluation questions which have been left out or picking up previously solved questions when interrupting the module, eg. when closing the window and continuing the module at a later time.

Example 1: I have 10 questions in my quiz, I solve question 1-8 and 10, I leave Q9 empty, I reach a score of 90%. When reviewing, all are correct except Q9, good.

Example 2: I solve the module, I close the browser after Q8 to finish later. I reopen the module, I continue with Q9 and Q10. I reach 20%, when reviewing, Q1-8 are marked incorrect (although correct choices remembered and still ticked), only Q9-10 are marked correct and are also correctly evaluated. When opening again to revisit and review, it's all garbage and mixed up as per above, also Q9 and Q10.

Example 3: Same as example 2, I solve Q9 and Q10 but also quickly visit Q1-8. I only toggle through the slides, I don't change any inputs though. I reach 100%. When reopening to revisit, everything is in order.

Good morning Joël!

Thanks very much for the detailed information. I'm happy to hear that you were able to make some progress with this. I do see that Jonathan is still testing the course. 

I'm going to pass this information along to him, so he's aware of the steps you've taken, what you've been able to fix and what problems are still exisiting with the course. 

I'll keep an eye on this thread, just in case you or Jonathan find any additional information.

Thanks again for keeping me updated!

Christine

Hi,

I have also encounter a few oddities or bugs where it comes and go unexpectedly whereby some layer eyes of the base layer I have set it closed whether in review layer/ correct and incorrect layer, but when I tested out more than a few times of re-try quiz and review quiz randomly without following common sequence of the project flow, some of the layers eyes still open and some buttons cannot be click or dragged. It was like the mess is all over the place.

I'm thinking probably it was because of the quiz which is part of the question bank and after each quiz has its own review slide which actually links to the final review slide to determine final score. Or could because of too many triggers set?

I cant share the file, like I say "the oddities or bugs comes and go unexpectedly"

Did anyone experience this?

Regards,

Emalyn

Hi Emalyn,

Since you mentioned that issues come and go and are not always reproducible, I'd suggest starting with some easy troubleshooting first:

• Work on your local drive (your C: drive).Working on a network drive or a USB drive can cause erratic behavior, includingfile corruption, loss of audio, and other unexpected behavior. 
• You should also make sure the directory pathto your project files and your published output is less than 260 characters(for example C:\Articulate).
• Avoid using special characters, accents orsymbols in your file names.
• If it's only in this one file, try importing it into a new Storyline project
• If it's happening across all projects, conduct a repair of Storyline
• See if the behavior continues when you publish and test in the intended environment

If you're able to replicate the behavior consistently and you'd like us to take a look at it, please share your .story file here or send along privately to our team. 

Hi Ashley,

Re-importing story file did work so far.

Thanks!

Cheers,

Emalyn

Thanks for the update Emalyn!