How do you evaluate the effectiveness of a course?

Oct 28, 2013

A colleague recently asked me to evaluate an instructor-led course for effectiveness.  Coming from the instructor-led/ADDIE instructional design side, I have a list of things I look for, from content to support the learning objectives, instructional variety, and a mix of exercise pairings.

When evaluating an elearning course for effectiveness, what do you look for ?

6 Replies
Cary Glenn

What I look for is long term changes in behavior after the course. A type 3 evaluation from Kirkpatrick. If depends on the course how I will do this. I would look at the objectives and see if people are still able to do or remember what the course taught them weeks later. Depending on the situation, I may have to send out a survey to the managers, visit a site, look at safety reports, or look at HR reports. This isn't easy and often the results are difficult to interpret. There are many factors in changing behaviors/skills/attitudes and a couse is only one of them, so the course may have changed the person or some other factor may have done it.

Sheila Bulthuis

I agree with Cary in terms of the "ideal" formal evaluation. 

However, Doreen, if you're asking about how we, as ISDs, look at a course, the factors that are important for an e-learning course to be effective, I think it's the same as for ILT: whether the learning objectives are accurate; whether the content supports/aligns with those learning objectives; how the content is organized (does it make sense? flow, build on itself?); level and type of variety in the way content is presented; types of interactions/exercises and their relationship to objectives. 

Learning is learning, whether the medium is face-to-face or online.  =)

Doreen Rambke-Hartz

Thanks for the responses.  I'm looking at this from the ISD perspective, so along the lines of Sheila's response.  I think I'm asking, "is this course design effective."  Many times I'm asked "Is this course good?"  It might be a vendor course, an internal course under development, etc.  

I've seen some internal courses that others have 'designed' and found they really don't know what they're doing from an ISD perspective - the objectives aren't tied to the content, the learning objective is to 'demonstrate xxx', yet there's no activity or practice that allows the learners to do that.  Many were designed from the perspective of what the software can do (if you load your content here, it will create a clickable wheel, etc.). 

Which is why I really like Articulate - it's a blank slate from an ISD perspective.  I'm still curious what others do to check the ISD effectiveness of a course. 

Nicole Legault

Hi Doreen, 

Interesting topic, thanks for posting about this!!

Here are some things I would be looking for when checking the effectiveness of a course:

  • Was the navigation clear and simple to understand? 
  • Did I always know where and when I needed to click?
  • Were clear and proper instructions provided?
  • Was there too much text on the slides?
  • Was the button placement consistent?
  • Was the material laid out in an order that makes sense? 
  • Was the material confusing or orderly?
  • Did the quizzes cover an appropriate amount of content?
  • Is the media (video, images) used for an instructional purpose?

Just a few things that I think contribute to the overall effectiveness of a course, I think some of these are instructional design, or ISD based questions, and some are maybe less so. All good food for thought when reviewing a course though. Again, great topic, hope it gets more input from the rest of the community ! =)

Eimear O Neill

Doreen, 

I customised Cathy Moore's "Checklist for Strong Elearning" from her invaluable Elearning Blueprint website!!! 

I send it out to other IDs who act as my first reviewers.

I use a score system 1 to 5.

It deals with questions around

- visible measureable goal,

- on the job skills and need to know information,

- interactivity,

- learning flow,

- level of previous knowledge and expected reading level,

- activities that reflect real life on the job challenges, skills & and knowledge,

- realistic consequences in feedback,

- reference material, 

- believeable characters and challenges,

- believeable visuals,

- use of narration,

- writing style. 

Basically you are ensuring the experience is matching your objectives and it is instructionally sound.

I recommend you check out Cathys website at http://elearningblueprint.com/ you'll get way more than a checklist for your money but a process on how to "design smart elearning for real world change"!! A must have resource that works for me everytime

Good luck!

This discussion is closed. You can start a new discussion or contact Articulate Support.