How can I better set expectations with my SMEs?

Hi Articulate Community!

I've been struggling with some recent projects. Not so much from the design side but from my stakeholders and SME side. Half-way through the projects, I'm hearing from them that the content in the course isn't relevant or even accurate in some places.

I always send the PowerPoint slides over for review before I begin recording and animating and building quizzes. What else can I do? I'm not sure if they're changing their minds or if they aren't reviewing the files I send to them, but we're losing time and everyone is getting frustrated. These are senior execs in some cases so I can't make them read back every slide to me. Seriously in need of some ideas and guidance.

Thanks!

10 Replies
Christine James

Hello Jay

I create a service level agreement upfront.  They know that the content they approve is what is delivered.

I also try not to send them too much information at once.  keep them in the loop but only get them involved when they have to do something.

Also, set clear deadlines and expectations.

Shawn Stiles

One of the things I started doing wth my SMEs was to hold a kick off meeting to discuss what the project was about and the roles/responsibilites of everyone invovled. I try very hard to help them understand that what we do isn't magic and that it takes time to create training.   Here is a presentation that I use with them, I find that it helps.

Wayne Vermillion

Jay, you need a sufficiently large shark on your side to counterweigh the senior exec SMEs, I think. I would take slide 16, especially the lower bullets, from Shawn's preso to that person and show that you're not getting the reviews you need.

Now that you've identified the problem, get the shark to commit to your solution. Were it me, I would build on Slides 8-11 in Shawn's preso with idiot-proof applications of Bloom's Taxonomy. Give them fill-in-the-blank options for objectives using Bloom's measurable verbs - no excuses for not having relevant, measurable building blocks then. You can also point out that this is the very best use of their time, that you've met the SMEs more than halfway with something simple and easy, rather than starting to re-interview and re-define.

Shawn Stiles

Wayne, I like your idea of the shark, getting someone who is senior enough to drive the SME's to comply is important. On the other hand if you are not getting the feedback that is needed or the participation required to be successful then going back to the project sponsor and having a frank conversation with them about the lack of support/feedback and then bring up the question of is the project really that important in the first place. I've found that most of my more successful projects have has SMEs that really grasp the importance of what we are working on and make the extra effort to make it work. 

Dave Neuweiler

On the relevance issue, and I may be preaching to the choir here, but it may simply be a matter of not establishing training objectives.

Consider:

 Objectives are the glue that holds training together. Many times, though we see people champing at the bit to start writing the content of a program before performing a task analysis and deriving concise objectives.

Performance objectives describe what the trainee will be able to do at the successful completion of the training. In their absence, ask yourself these questions:

Without performance objectives, how will the developer derive accurate and appropriate content?

(Answer: he or she won't.)

Without performance objectives, how will the developer know what the training is expected to accomplish?

(Answer: he or she can't.)

Without performance objectives, how will the students know when they've mastered the instruction?

(Answer: they won't.)

In short, and just my opinion, I think you should look at two things.

First, did the stakeholders/SMEs establish the objectives to help you develop the content? If not, then this may be the source of their unease.

Second, if reasonable objectives were established, does the content point to one or more of them? If some of the content does not, then ask yourself why it should be there. (I've never known an instructional designer -- myself included -- who didn't sometimes go off on a tangent.)

The other part of the issue was that some of the content wasn't accurate. I'd ask two questions:

1. What needs to change to make it accurate (after all, that's what SMEs are for)?

2. Did something change from the initial review that made the information inaccurate (it's either that, or the SMEs didn't review what you sent them the first time)?

Now, how you frame those questions depends on company culture, individual egos and the like. But it's important to learn the answers, don't you think?

I hope that helps, and good luck moving forward!

Steve Flowers

One of the ways I do this is by making sure they are in the balloon basket with me for every mile of the journey. If the balloon crashes and burns... we're all in that basket together. SME's are awesome navigators, without which I would many times be dead in the water. Where the SME is concerned here's how I see my job as an ISD:

  • I help triage priority (help separate the need to know from the nice to know)
  • I help establish solid goals and objectives.
  • I facilitate "gardening" (letting things grow when it seems healthy, then pruning back when it's appropriate. I enjoy bringing multiple SME's into the same session. I learn far more from banter and discourse than I would drinking from a single SME firehose)
  • I advocate for the learner population (SME's know everything... Everyone else doesn't necessarily need to)

Interacting with an SME is a dance. It's a relationship that requires effort to maintain. If your SME's are criticizing your content scope, something came apart at the seams somewhere along the journey.

I've been involved with many projects that were "packaged information conveyance". If I can influence the stakeholders to stay away from that goal, I will. One of the strategies for this is to offer "supplements" to the training. These are documents that are attached as resources. They aren't core to the message. They aren't required. But they are still there. Most SME's feel much better that they were listened to as long as that information has been included.

I posted this to the old forums (http://www.articulate.com/forums/general-discussion/21244-process-developing-e-course.html). This is the approach I've used with the most success I've bolded the prime step that you might have used to avoid downstream misalignment below:

These are our four prime phases:

  • Decision: This is our performance analysis / pre-design analysis / cost benefit calculation / media selection phase.
  • Development: Here we establish the refined solution design and complete the engineering and assembly of the solution
  • Solution Evaluation: The summative testing of the solution against known gap variables.
  • Maintenance: We try to address maintenance planning up front in the cost benefit analysis. This phase is intentional and is taken as seriously as the initial three phases.


Specific to the development phase we have several key deliverables that support the process:

  • 1 - Design Document. This contains solution framing variables, description of performance gap, audience definition, performance objectives and high level scoping expectations. This is a brief document that forms the foundation of the rest of the development outputs.
  • 2. - Assessment Elements. A document that defines all assessment activities, test questions. Typically we see vendors wanting to deliver this at the end and it ends up being a trivia culling from the content. We require these items up front. How are we going to measure the gap fill?
  • 3 - Content scope. A document that defines all of the topic, content, task, and concept elements that will be included in the course. This is, in part, an exercise in controlling the vendor / developer's risk. We expect this to be approved before any solution blueprinting is pursued.
  • 4- Structural flow. This can manifest as an outline series or flowchart. This represents the high level instructional architecture and content sequencing. This also articulates any instructional scaffolding strategies, outlines / illustrates patterns and activity elements projected for application within the solution.
  • 5 - Prototype storyboard. We want to get a feel for the vendor's abilities in narrative and experience mapping. This establishes a small slice of the delivery for review. This is risk mitigation for us.
  • 6 - Prototype solution. This maps to the prototype storyboard and serves a couple of purposes. We want to see how well the vendor translates their prototype vision. We also want to provide a sectioned reference framework for the SME's during storyboard review.
  • 7 - Storyboards. The build blueprint.
  • 8 - Draft solution. This is the first release of the final product.
  • 9 - Discrepancy report. Customer delivery.
  • 10 - Final solution. Discrepancies addressed. Source materials and final delivery.

You may also find this article interesting (from Tom Kuhlmann), the comment responses are also pretty snazzy:

http://www.articulate.com/rapid-elearning/heres-how-to-help-your-subject-matter-experts-build-better-e-learning-courses/

 

Two of the comments, in particular, stand out for me:

 

"I try to get my SME to talk to me, as a real person, perhaps on the phone, instead of writing it all down. Record it (I use a handheld digital recorder) and summarize for them. Few people like to see edits to their written work. I also use case studies or stories of real people/situations often.

 

I just finished interviewing managers of online professional development programs for educators from across the country. One great tip I learned was including a third person on the design team as a manager, perhaps including someone from the funding agency. That way, there’s always a 3-way vote, not just a 50/50 split."

 

Kathy Sierra has written some really dynamite books on IT subjects:

 

" We try to get our SMEs to explain the topics with a whiteboard or just a napkin, to a person who is squarely in the target audience. It is only when they see the confused look on someone’s face, or the requests for clarification that they start adding in the really good stuff… Those alternate ways of thinking about it, better analogies, etc. Then we take photos of the whiteboard and napkin drawings, and these are often what end up in our books.

 

... by far the best for us was having a real person for them to “teach”. We found it too hard to convince some SMEs that we (course devs / IDs) actually KNEW what would be too confusing, but having just one person they approved as an Actual Target Audience person extracted the right things out of them… All of those “well let me try it another way” secondary explanations and FAQs that are so hard to get at otherwise."

 

David Becker

A quick and dirty solution:

Explain process as follows up front to SME's:

  • Here is my approach and milestones. Please ensure you set aside time for review at these milestones. I recommend x hours.
  • Once objectives and high level approach is agreed, I will require sign off from all SME's
  • At draft storyboard, you will have opportunity to change text/media etc however you want. Once agreed I will require sign off
  • At Alpha build, you can change minor text edits, static images etc, add next content, remove content etc. Once agreed I will require sign off
  • Once it goes live, we will enter a 6 monthly maintenance cycle, any changes requested after go live will be recorded for next scheduled maintenance.
Steve Flowers

David's approach is similar to mine. I call this the "states of matter"

It starts as gas > moves to liquid > to Jell-o > to wet concrete > to set concrete. The further down the road we go... the harder it's going to be to change and I tell the stakeholders "you need to know that it's not free or easy to chip away concrete!"

Kendra Barker

Set expectation early in the process. 

Explain in detail what is required by them and when (firm deadlines).

Learn a little bit about their personal life and their work style. Open up to them too.

Remind them that they are part of a team to create an outstanding course to help others.

Ask what is the best way to get a hold of them for questions? (email, phone call)

Ask them what days are best to contact them.

Tell them how much they are appreciated.