Follow These 3 Tips to Put the “Pro” Back In Your Review Process

If you ask a group of e-learning designer/developers to discuss the hardest parts of their role, odds are good that someone will bring up the topic of course reviewers (SMEs) and feedback. From reviewers who confuse and contradict, to those who fail to turn up with crucial information until the very last minute, managing the feedback process (and sometimes the reviewers!) can be a big challenge.

Articulate Hero Tracy Parish recently turned to the E-Learning Heroes Community for ideas with this challenge. Tracy was looking for suggestions on ways she could streamline her organization’s review process. Following is a summary of some of the community’s pro tips for making the review process more positive than painful.

Tip #1: Clarify Roles & Responsibilities Up Front

Often e-learning designer/developers find themselves stuck in the middle, trying to make sense of a barrage of feedback they can’t reconcile on their own.

One of the best ways to avoid getting stuck in this position is to conduct a project kick-off meeting where all the roles and responsibilities of the project team are clearly defined, including specifying who the decision-makers are and who will be responsible for collecting and reconciling conflicting feedback.

Holly MacDonald offered this suggestion and a tool that can help you clarify these roles and responsibilities up front: “...there should only be one client and they are accountable for the content. Maybe a RACI chart? I find these are helpful to frame discussion about ‘what will we do if...’ at the early part of the project.”

By taking some time up front to clarify roles and responsibilities you’ll be in a much better position to facilitate conversations between reviewers with a clear escalation path for decision-making.

Tip #2: Educate Your Reviewers

I’ve found that when we talk about the review process, it’s easy to get too focused on the mechanics—which spreadsheet to use or what date we need the feedback by—and lose sight of the what—i.e., what kind of feedback you’re looking for at each point in the process.

Bugs vs. Changes

When it comes to the kind of feedback, one thing that trips people up is  confusion about the difference between a design change (typically a preference) versus feedback on a design defect (a bug)—and how that distinction will impact your priorities differently at various phases of the project.

To help clarify the difference between bugs and changes, here’s one example I’ve used:

I’ve found this example is simple and relatable and gives everyone the chance to agree on the language so we can move on to talking about how we’ll deal with different types of feedback and the various milestones.

Holly MacDonald beautifully summarized this point: “Categorize...edits versus changes...so they understand that wholesale changes are ok when things are still early in the development phase, but after that you are looking for refinement. Clearly re-iterate that there are types of review. Perhaps create a spreadsheet with categories of feedback.”

Cost vs. Value

Sometimes a little proactive client education is all you need to help your reviewers get on the same page with the project priorities—and sometimes you need to take it a step further. When reviewers draw a line in the sand that threatens the successful launch of the project, you may need to reframe the conversation. Several community members chimed in with the suggestion to give reviewers a little cost vs. value reality check by assigning their changes a dollar amount, e.g., "If I was charging by the hour, these changes would cost $xxx." Using this type of a reality check can give everyone a clearer understanding of the impacts of their changes before they actually impact the bottom line.

Tip #3: Make the Review Process More Transparent

One of Tracy’s big pain points was that busy reviewers endlessly contradicted each other. To avoid this frustrating back and forth some community members find it best to pull everyone together for a face-to-face review. For example, Mohammad Hassam finds it best to pull everyone together for a face-to-face project review: “Schedule a meeting with all of them at once. Invite your client, reviewers, SMEs and then review your course. I found it much better because, except graphics and layout, if anyone has any view in terms of content, assets or materials...the other person sitting next to him will most probably object [to] it. The conversation between them keeps going until they come up with certain ground rules. You have to list down what's being discussed, and their comments (note their names too), and send a copy via email after meeting.”

Other community members felt that large face-to-face review meetings weren’t always practical. Instead, they shared their experiences using technology to bring more visibility to the entire design, development, and review process. Some community members use a tool, reviewmyelearning.com, to support their release and review cycles. Others, like Karlis Sprogis, use a combination of free or low-cost tools like Google docs, Dropbox, Trello, Join.me, and Invisionapp.

And Jane Manduke offered this comprehensive process along with the tools she uses: “I create a page on my Wordpress website and upload the course. (I make sure the page remains hidden from all menus, tag-clouds, etc., i.e. you won't find it unless you have the link.)

I send all reviewers the link. Usually I stipulate in my contract that there be one point of contact on the client side and she sends it to the team. When I send the link, I request they post comments on that Wordpress page. (This doesn't always work; some people aren't comfortable posting comments, or maybe they just don't have time. But I have had some good conversations that way.)

I give reviewers a day or two to look at the prototype/build version and set a time and date for a 'consolidated' review.

The consolidated review is handled remotely, at my end, through Join.me, a web-conferencing and screen sharing tool. All team members/stakeholders are invited. We walk through the e-learning step by step. I collect their feedback over the phone.”

Whether you use an assortment of high-tech tools or stick with good ole’ fashioned face time to facilitate reviews, one thing is clear: making your process more visible shows what really goes into creating e-learning and gives reviewers the tools and information to spot redundant or conflicting comments so they can be addressed (by experts who are equipped to debate the minutiae) before they reach your to do list.

Summary

The feedback process is fundamentally a project management challenge. Project management is one of the many duties of e-learning designer/developers and like all of your other challenges, it’s bested with a dose of preparedness and plenty of practice:

• Think like a project manager. Before jumping into course development, plan some time to clarify roles and responsibilities on your project team. Sometimes our knee-jerk reaction to a lack of clarity or clear accountability is to create MORE process—which may make your job even more complicated. Instead, try starting with a good conversation to set expectations and get everyone on the same page.
• Act like a consultant. Consultants are constantly educating their clients. To help you educate your reviewers, create a brief (one page) handout or infographic that walks through each phase of your design and development process and explains the type of feedback you’re looking for (and from whom) in each phase.
• Do add tools to help foster more transparency. At a minimum, consider a shared resource for gathering review feedback and tracking action items.

Resources

Check out these community discussions and resources you can use to put the “Pro” back in your review process.

E-Learning Client Management: Tips for Dealing with Unhappy Clients

4 Easy Ways to Win Over Reluctant SMEs

Help! How to Get SME Feedback on Time

What golden nuggets of wisdom have you found most helpful for your review process? Jump into the conversation and share your ideas with us! And while you’re at it, we’d love for you to follow us on Twitter where we post the latest and greatest news about everything e-learning.