Narrated Quizzes

Hi Russ,

I agree with you that narration is invaluable when learning drug names and terminology and  I like the fact that you are using e-learning to remedy the problem of similar sounding drugs.  I noticed in your voiced version that the yes and no options changed positions; as long as I chose the second option, I was correct.  I would suggest that you keep the yes and no in the same location throughout the quiz, so there are no hints, or perceived hints.  

Also, if you can make the scenario more realistic (use peoples faces and give them an identity)  as well as showing the consequences of their mistakes - you can increase the quiz takers pressure to perform and more closely simulate real-life scenarios.  i.e. Mrs. Marizpan, aged 81, has suffered a stroke and is on blood thinning medication.  One of the drugs you have on your tray is Coumadin - is it safe to give it to her?  If they chose the right answer - the patient looks better, if they chose the wrong answer show them what the consequence of their wrong choice would be (i.e. cardiac arrest, death, etc.)

Just my 2 cents.

Linda

Russ, I totally failed your quiz, LOL.  I agree with Linda, above, about the Yes/No switch.  To the question - I traditionally turned off the narration during quizzes, but when dealing with difficult subject matter, I believe the audio should be available and the user should always have the option of turning it on/off.  In fact, I have started designing all of my courses with optional audio, because as a learner, I do not like having to listen to the audio unless absolutely necessary.

Might I also suggest a subtle prompt or even some sort of dictionary attachment for your drug quiz?  That way if they want to sneak a peek at a reference, it is available and handy at the top of the screen. 

I like the story questions, but I would also throw in some multiple choice where they had to choose WHICH drug should be used, instead of just Yes/No.

Fun quiz!

(Hi Denise, it's your "twin", Jenise Cook! LOL)

Hello, Russ!

I really like the photo images of the drugs on each quiz page.

And I, too, prefer not to have audio narration playing during the assessments because it interrupts my concentration and problem solving. I don't have specific research in front of me right now, but I recommend Dr. Ruth Clark's and Dr. Richard Mayer's "e-Learning and the Science of Instruction" for brain-based, adult-learning research. Dr. Karl Kapp also references them here:

http://karlkapp.blogspot.com/2009/11/audio-in-e-learning.html

A couple of years ago, for a course I helped develop for the National Assoc. of Pediatric Nurse Practitioners, we used an Engage interaction for the pronounciation of the drugs/medications. We chose the FAQ interaction as our list of medications was not a long one (under a dozen). We "spoke" the pronounciation within the sentence context of a physician or Nurse Practitioner discussing the drug with the patient (or guardian).

For a longer list, I propose using the Engage Glossary interaction. Either way, you could use the Presenter course player in Slide Only view, and have the interaction as a Player tab in the upper right corner (near the Exit icon).

But, If you're only using Quizmaker, I agree with Linda's idea above. Take advantage of QM's Timeline feature and Blank Slide feature. You could precede each Yes/No assessment with the Patient/Nurse scenario created on a Blank Slide.

@jenisecook

Hey, sis!  (waving at Jenise)  How funny that our names are so similar and we're in the same field, lol!

I want to second your point that using narration for pronunciation is a big plus for online training.  I think the training subjects would sincerely appreciate having that extra feature for this and other similar tests.  Before rapid authoring, I did a training manual for medical terminology and even though it was delivered on a local computer, I remember us lamenting that we wish we could have provided an audio based dictionary to accompany it.  The difference in pronunciation might make a huge difference in the correct response.

(Waving back at Denise, my "twin"!)

Agree with your points, and it's important to hire an experienced voice-over actor/talent who may specialize in medical/pharma content.

On a side note: With the pharmaceutical commercials on TV, my husband and I learn how to pronouce all of those different names. Some of the names are so "different" that we created a new TV series in our imaginations to entertain us during those commercials (when our show isn't recorded and we can't fast forward through the ads).

For example, there's the new TV series about extra-terrestrials we've imagined starring the alien Levitra (a female) from the planet Propecia.

Off topic, but we sure remember how to pronounce those medications! (Dang if we remember what those drugs treat, tho. LOL)

Thanks all for taking the time to review the quiz prototype and make suggestions. Here are some of my thoughts regarding your comments.

Yes - No: By habit I have the responses randomized but with two options. It probably makes more sense to keep them constant.

Most nursing quizzes make use of scenarios with fake names, ages and background information. In my opinion, those may be fine for elearning simulations, but are they really necessary for effective quizzes? Or is the additional information simply noise and clutter?

To Denise 1: This is one of about 20 quizzes we have on medications. In some of our other quizzes, we have multiple choice questions along with links in answers to drugs (via MedLine Plus). See this quiz as an example. The focus for this particular quiz is on whether narration is of any assistance to learning. As for being able to switch audio on or off, I have NOT been able to find this feature in QuizMaker.

To Denise 2: According to our narrator (who has extensive experience in the pharmaceutical industry), the pictures of drugs are as important as the narration. I would like to be able to show a picture of the "correct" drug in the feedback response, however at present QM does not have that capability. Yes, I could use branching but then our quiz goes from 75 to 225 slides. The other problem with this approach is that we limit our quizzes to 30 questions using a random selection processs. Unfortunately, if I were to set QM to select 30 slides, there is no guarantee that the proper linked branched slides would be selected.

I got the idea for narration from a nursing textbook on anatomy that had a CD-ROM included with all the pronounciations. I wanted to try this with medications since I already had the quiz and the images. 

The other major challenge that I face with our type of nursing quizzes is getting access to affordable and royalty-free medical images. Any sources you can suggest?

Agains thanks for you comments / feedback.

Russ 

worryingly I got it all right, and know nothing about drugs!  (I like the audio if that helps) it would help me learn. 

For anyone interested in seeing the final product, the fully narrated Right Drug Quiz has been developed and posted on our public Learning Nurse website. It was a lot more work than I expected. For a quiz with 75 questions, you would have to narrate and create 225 audio files (one for the question, and one each for the right and wrong answer feedback). So you need to make sure the extra effort, time, costs and larger file sizes actually contribute to enhanced learning. In our application, hearing the correct pronunciations of the brand and generic names of drugs that are often confused, does seem to contribute to effective learning (and hopefully reduce medication administration errors). 

I would appreciate hearing about any further insights, applications and experiences with narrated quizzes. Thanks.

Russ 

Hi Russ,

Thank you for your update. On the pragmatic side, when I read your details, my internal calculator started spinning.

75 QM quizzes (30-60 mins each quiz to design/develop) = 37.5 hours up to 75 hours work effort

For the audio work, I'd take the QM effort and multiply that by 2 at minimum by 4 at maximum for anywhere from 75 hours to 150 hours for recording, editing, compressing, and importing into QM.

37.5 + 75 = 112.5 work effort hours (3 weeks approx/based on 40 hour work week)

37.5 + 150 = 187.5 work effort hours (over 4.5 weeks based on 40)

75 hours + 75 = 150 work effort hours (3.75 weeks)

75 hours + 150 = 225 work effort hours (5.5 weeks, approx.)

How close am I to your actual work effort?

The reason I wrote the above is because we are frequently asked, "How long will it take?" followed by "How much will it cost?"

For project costs, an independent consultant would take either 112.5 hours or 225 hours and multiply that by the hourly rate he/she would charge the client.

Just curious.

@jenisecook

Hi Denise,

Excellent question ... and one that I'm sure many new quiz developers are asking as well, especially if they have to submit a proposal or bid. The Right Drug Quiz (74 questions) was done for our own use, so I did not track actual hours. However, to the best of my recollection, here is the approximate effort that I had to put into this narrated quiz project.

1. Get list of confused drug names from Institute of Safe Medication Practices (ismp.org) – 1 hour

2. Look up and document what each confused drug is used for, and its generic name (from MedlinePlus) – 8 hours

3. Find and copy images of each drug (with permission from Drugs.com) – 6 hours

4. Create the quiz (74 questions) in Word – 7 hours

5. Prepare the narration script (4 revisions) – 5 hours

6. Contract out the narration ($600)

7. Copy and paste quiz questions from Word to quiz program – 4 hours

8. Add drug images to quiz program – 3 hours

9. Edit audio provided by professional narrator into 148 question and answer audio files – 4 hours

10. Insert audio files into quiz – 3 hours

11. Preview quiz to make sure that everything is working properly – 2 hours

12. Publish, enhance code and upload to website – 1 hour

The actual time to develop this quiz was between 40 to 50 hours. However, a few points to keep in mind. First, this is fairly simple factual content and that no content specialists were involved. Also, I have done hundreds of quizzes so have my procedures and systems well established.

This quiz took more effort and time than my typical quizzes. I can usually create a Learning Nurse quiz in one day (7 hours), and take another half day to copy and paste it to the quiz program, publish and upload it.

Narrated quizzes take much more time and effort than the traditional text / image quizzes. So you better be sure that the results are worth it.

Unfortunately, our very early results are disappointing, i.e. the 25 narrated quiz questions should take about 15 to 20 minutes to complete. Our early data shows that most users are doing the quiz from 5 to 10 minutes. This suggests that they are NOT listening to the full narration, but are clicking on "Continue" as soon as they get the answer. If this is the case, why bother with the narration?

I will continue to monitor the results to see if this trend continues with a larger sample. So for now, a word of caution ... before you invest a lot of time and effort into developing narrated quzzes, be sure that your learners want and will make use of the audio component! Test! Test! and Test!

Russ