Behavioral Evaluation Methods

Hi JM,

Kirkpatrick's levels are accepted pretty much internationally as an evaluation model. Apologies if you know this already:

Level 1: learners satisfaction with the learning experience

Level 2: learners demonstration of knowledge/attitude, usually by taking a test of some kind

Level 3: learners behavior changes in the workplace

Level 4: workplace business benefit following learners' behavior changes

There is a problem for instructional designers. Within a training course or an elearning package they can build in tests, which provide level 2 data. But generally to get Level 3 data the learning intervention needs to be connected to other systems. Two examples:

• If it was compliance training, which for example required operators to check on control machinery every so often, or perhaps wash hands on a regular basis, these activities can be monitored electronically when they happen, and provide real data evaluating behavioral change.
• If it was an interpersonal skill, which for example required staff to talk to customers in the best way, to evaluate behavior you would need to get feedback from customers, and/or records of observation of that behavior by peers (or more usefully) the managers/team leaders of the people learning.

The 'elephant in the room' here is that even before we look at the evaluation models, as a society we are not very good at encouraging behavior change and organizations mistakenly 'delegate' it to instructional designers. If the learners are highly motivated to change, then this will be OK. But if learners are very busy, or management are trying to get them 'out of their comfort zones' then this delegation typically fails.

In reality, many learners fall somewhere in the middle, and there are things that can be done, as in the examples (simplistic examples admittedly but just for this short post). My major interest is in connecting learning to real work, and generating the conditions needed to get real behavior change (and evaluation!). By all means send me a PM, I am happy to talk more, or continue this thread.

JaF

Thanks John & Nick.

John, I am aware of Kirkpatrick training evaluation model. But just wanted to check if there is any criteria or specific parameters to evaluate trainees behaviorally?

Regards,

JM

This may seem a slightly trite answer, however, the criteria are the ones set by the organisation to represent "success" following the training, (and the state they are at before training, which have deemed the training necessary).

I think.

Just to say what Bruce has said in a slightly different way, normally when you design training, you do so against a set of learning objectives.This is fairly easy when it's, for example, training in Excel. If the company needs that all admin staff can use Excel to enter data and build charts for use in client presentations - well, you have criteria there for the objectives of your courses, and for evaluation: it's easy to test peoples' Excel skills to see if they can do this.

For behavioral change it's a bit trickier, partly because organisations often do not express what they want overtly, in an objective and written way, they just know what they don't like. But sometimes they do, in lists of 'competencies' and so on.

Criteria for evaluation of behavioral skills are based on what people do and do not do in response to typical situations. For example if you are training desk reception staff dealing with irate customers. For example, you might say that in response to such a customer, successfully trained people will: Not lose temper or display annoyance at any time; focus on facts; ask questions to identify the real problem....etc etc. All of these things can be observed, and thence evaluated.

JaF