Resaerch Results that Compare the Effectiveness of Training Types

28 May 2016 by unknown

The approach to scenario design that I suggest in my book and courses borrows ideas from problem-based learning, guided discovery, productive failure, and similar approaches. Here’s some research support for these ideas.

Give them an activity, not a presentation

One of the major concerns I’ve heard from clients is that it’s “unfair” to plunge people into a well-designed problem, such as sending them directly into a scenario without first telling them everything they might need to know. Another common argument is, “They’ll make mistakes and only remember the mistakes, so then they’ll do it wrong on the job.” As a result, clients want to present everything first and hold people’s hands. So first here’s some research about that.

A good first place to look is Make It Stick, a book that summarizes learning research. Some quotes with their Kindle locations:

“Trying to solve a problem before being taught the solution leads to better learning, even when errors are made in the attempt.” p. 4 location 107

“When learners commit errors and are given corrective feedback, the errors are not learned. Even strategies that are highly likely to result in errors, like asking someone to try to solve a problem before being shown how to do it, produce stronger learning and retention of the correct information than more passive learning strategies, provided there is corrective feedback.” p 90 loc 1265

“It’s not the failure that’s desirable, it’s the dauntless effort despite the risks, the discovery of what works and what doesn’t that sometimes only failure can reveal. It’s trusting that trying to solve a puzzle serves us better than being spoon-fed the solution, even if we fall short in our first attempts at an answer.” p 94 loc 1311

“Elizabeth and Robert Bjork, who coined the phrase ‘desirable difficulties,’ write that difficulties are desirable because ‘they trigger encoding and retrieval processes that support learning, comprehension, and remembering.” p 96 loc 1373

“To be desirable, a difficulty must be something learners can overcome through increased effort.” p 99 loc 1381

There’s a lot more in the book, including citations.

This blog post links to research that supports productive failure as a way to support transfer, not just quick regurgitation.

A lot of the research focuses just on whether people remember the stuff they learned, but (like the study above) some look at whether people can apply it in a new situation, which is what we want.

Scenarios and transfer

Ruth Clark also cites research in her book Scenario-based e-Learning: Evidence-Based Guidelines for Online Workforce Learning, especially in chapter 10. She includes suggestions for when productive failure is probably most effective (basically, for people with some pre-existing knowledge and well-structured problems) and argues for scaffolding and guided discovery, which I’ve translated into giving links to supporting information in the activity and organizing activities so they progressively build and reinforce skills. She also highlights how scenario-based learning appears to help people transfer what they learned to a new task and points out that problem-based learning seems to be more motivating.

Clark Quinn also describes research that suggests that problem-based learning is better for long-term retention and skill development.

Here’s another meta-analysis that also seems to support the use of PBL in medical training.

In this summary of points made in a presentation, Karl Kapp links to research that suggests that people who used simulations feel more confident that they can do the thing in the real world. Of course, that doesn’t mean they’ll actually do it, but it’s a start. He also references studies that seem to suggest that “active learning” in a simulation has better results than traditional presentation-driven learning.

Will Thalheimer has a PDF summary of research into the efficacy of scenarios, with a focus on how they appear to improve knowledge retention and retrieval.

Clients’ concern about research might mask other concerns

Sometimes clients resist scenarios because they want everyone to be equally “exposed” to the same information. They might disguise this as a concern about whether research shows that scenarios work. If this appears to be the case, you could still argue for a scenario-based approach but make clear that in the feedback for each choice, you’ll include the information that people must be “exposed” to. Whether they choose correctly or incorrectly, they all will be faithfully and accurately exposed. They’ll also have a (probably) better chance of remembering and applying the information later, since their exposure took place in a problem-solving, realistic context, not in a passive presentation.

Unfortunately, a lot of the research into scenarios, simulations, and guided discovery has been done on people who are supposed to apply academic knowledge and not, say, change how they talk to customers. I consider this a drawback because we want to change behavior, not knowledge, but most of our clients are still focused on knowledge, so citing the research should actually help you include scenarios and see for yourself whether they change what your audience does.