25 Replies
El Burgaluva

Sid,

At the very least, you might want to outline a few focus questions.This will serve a few purposes:

1. It will force you to reevaluate whether you've achieved your objectives before sending for review 

2. It will give the stakeholders and SMEs a starting-point for discussion and feedback

...and...related to #2... **drumroll*

3. It will -- hopefully -- prevent them from just giving a random list of comments as long as your arm, which is what's likely to happen if you don't give them a point of (limited) focus. This is because, if you don't provide focus and tell them what you expect, they won't know what to give feedback on; consequently, the default position will be to either give feedback on everything  in minute and painfully pontificating detail... or... feedback on nothing at all! Neither or which is helpful to you.

Hope that helps,

Leslie

Leah Hemeon

I agree with El. It's important to focus the feedback by reminding your reviewers what you're trying to accomplish. 

Also, to Poornima's point, a spreadsheet can be used to track edits. We have SharePoint in our organization so I set up a tracking list for each course that goes for review and then reviewers can add comments, even attachments like screenshots or other documents. 

In addition I always send a Word document version of the course so that reviewers have somewhere to make notes. I'll often send a link to the course out with the Word document attached to the email or available alongside the course in the LMS. I'll usually set up a meeting with the reviewers about a week after I send the course for review to gather their comments and go through the course with them. I find this especially helpful if this is the first time I'm working with that person. 

Bruce Graham

I normally send a link to the course on DropBox, and a version saved as a Word Document, with "Directors Notes" and guidance, as already mentioned.

When we do the second QA, (notmally takes 2 cycles), I re-emphasize that this is THEIR course, for THEIR customers, and they now need to make a go:no go decision on whether I have imterpreted and implemented their requirements correctly. Very often an ID will be blamed for any failure - so make sure you close that particular door to them.

I usually try to get ONE set of documents.

Bruce

Mike Taylor

I can't find it right now but Tom did a post on embedding a google doc to collect course comments/feedback.  I really like that approach and you could also use any survey tool like Survey Gizmo/Survey Monkey.  All of those automatically collect the feedback in a spreadsheet and allow you to make whatever fields you want (including drop-down lists for categories, etc.)

Instead of embedding you could also include a "Comments" link in the top menu or on every slide to do this too.

One of the benefits of this approach is that you can get comments from multiple people all at the same time and save it into a single document without any of the copy/paste combining of the data.

Tricia Ransom

In the past I created a Sharepoint site that had the courses loaded as well as a spreadsheet for their comments. I also included a sample of the type of comment I was looking for. In my email out, I would tell them that we (the training team) had already done a spelling/grammar/ check and that the course adhered to our development principles such as colors, graphics and things. In the spreadsheet, I had a column for my comments back to the reviewer where I would indicate if I had made the edits, or why I chose not to.

Oh yes, another thing - I ALWAYS put a big text box somewhere on each slide with the slide number. I take it out for the final draft, but it makes commenting/reviewing MUCH easier for everyone.

tin C

Thanks all for these great ways fo sending material to the SME for review.

I am still not being able to publish the course to a word document and see teh images. As far as I understadn all the images in each slide should be in the word document when publised to word? But I just get blank pages, without the images or text in them

Bob S

{Re-post from another thread}

This topic of the logisitics of SME/Stakeholder review seems to come up over and over.

In the advertising world there are lots of services/sites available that allow content to be reviewed, marked up, and the comments catalogued. Many even include review database and reviewer management features. I'm told this kind of remote review tool has become a "standard" amongst the best agencies that allow them to work remote artists/designers around the world.

Sadly there seems to be a hole for this kind when it comes to e-learning courses.

When I investigated this late last year, there were precious few options available to remotely review and comment on interactive Flash content (plenty for non-interactive Flash video). And only one I could find aimed directly at e-learning.

Seems to me there is a TREMENDOUS OPPORTUNITY for the folks at Articulate to leverage not only their authoring tool expertise but also LMS experience to offer this service...

Mike Taylor

Great thread here...has anyone ever used an 'project management' type tools for reviews, questions, tracking to items related to a course etc? 

Seems like tools along the lines of Asana, Wrike, Freed Camp etc might work? Anyone had success or failure along these lines?

I've briefly tried Dispatch.io and so far it looks pretty good. It enables comments on documents, etc from Google Drive, Dropbox, etc

https://dispatch.io/faq

onEnterFrame (James Kingsley)

Hey Sid,

Not sure if you found a solution for this… We were having issues getting good reviews from SMEs, stakeholders, etc. We tried a lot of the options mentioned here, Google Doc, Excel, Dropbox etc. In the end we created our own solution.  We can upload our courses to the cloud and send review invites to people. All the comments are collected in one place. Plus it has Screenr built into it. It’s been working really great for us and we decided to open it up to others.

You can check it out at ReviewMyElearning.com

Thanks

James

Bob S

OUTSTANDING!!!!!!

James, the product your team has developed looks outstanding. It potentially addresses soooo many issues.

No offense meant, but I was hoping the Articulate folks would be first to the table with this, but I'm very glad someone came up with the solution! My company will be reaching out to you shortly to learn more.

IMHO this solution deserves it's own thread becaue it truly fills a gaping hole that many developers/teams face.

onEnterFrame (James Kingsley)

Thank you Bob,

No offense taken. Although we (elearningenhanced.com) work primarily in Articulate productions we occasionally work in other's and wanted something that could handle those tools as well.

I'm glad you like it! We will be happy to answer any questions you have. 

Fiona  Telford-Sharp

That looks fantastic James. I was just having the conversation yesterday about how we are going to get reviewers to give their feedback. I will sign up for a free trial and experiment with it, but just a question - is it very easy to use for non-techy people? My course is for pharmacy assistants (drug store staff I believe you would say in the US lol), so they may be quite computer savvy or may be fairly computer illiterate. I need to make the feedback process as simple as possible. Have you had much feedback on whether your site is easy to use?

Scott Hewitt

Hi.

The review process!

At the start of project we discuss what we are going to review, when and how we are going to review it. Reviewing a prototype and then getting a set of word documents that we need to interpret can take a long time.

Some times we've used a tool like screen-o-matic and grabbed the screen with annotations. It has helped the developer and also we've also had a commentary and seen the course in action.

My experience is that get the review process into prototype as soon as possible. I've worked on projects where we've worked on screens and scripts that just have worked for the client when they have become 'interactive' and we've all wasted a lot of time - regardless of the review process that we've used.

On earlier projects we've built a script editor where everyone would work collaboratively which has been a success.  This was an internal app and sounds like http://www.reviewmyelearning.com/.

The main issue of marking and reviewing is that you are giving people 'flat' content and asking them to thinking 'interactively'. We try and work with a process of MVP - minimum viable product and do small prototyping. This makes the review process easier and allows people to review in the environment that the final delivery will be. It is not always possible - but where we can we try!