E-learning Vendor Evaluation Criteria

Aug 22, 2017


We have sent RFQ's to select an e-learning vendor for our organization. As part of the RFQ, we gave them a demo project to create 3 videos of different levels of complexity.

Now is the time to evaluate each of the vendor's work and sign the contract with one of them. Supply Chain team has asked me to come up with some technical criteria that will be used to select the vendor.

I came up with a draft of some criteria but I am not very satisfied with it. Since the demo that we have requested from the vendors is a video, should I base the criteria around videos or keep it generic?

Would really appreciate if I can get some suggestions on how I can improve the attached set of technical criteria.



7 Replies
Allison LaMotte

Hi Ashwin,

Great question! Is this criteria something you're going to use once? Or will you need to reuse it? If it's just for this one time, I think it makes sense to adapt the criteria to the specific exercise you requested from the vendors.

I had a look at your criteria and I think you've got the right idea in terms of what you're looking for, however I think I would personally use an Excel spreadsheet instead of a word document. That way you can set up all your criteria alongside the different vendors, making it easier to compare their scores.

Another thing I noticed is that there's a lot of subjective wording in your document, which may make it hard to evaluate the vendors objectively. For example, you wrote "user friendly navigation." How do you decide whether or not the navigation is user friendly? If 2 interfaces are user friendly, how do you decide which one is more user friendly than the other?

I would write definitions for each of your criteria that are as objective as possible, so you can rate each of them and come up with a total score. You could also decide to weight certain criteria if you think they're more important than others.

For example:

Rating on a scale of 1 to 5 (5 being high)

User-friendly navigation

5: I navigated through the course with no issue

4: I had 1 or 2 issues knowing where to click

3: I had 3-4 issues knowing where to click

2: I had 4-5 issues knowing where to click

1: I had more than 5 issues knowing where to click

Project Management Capability

5: I fully understand each step of their workflow.

4: I have a relatively clear understanding of their workflow.

3: I have a general sense of how their workflow works.

2: I know the big steps involved in their workflow, but not the details.

1: I don't know anything about their workflow.

Total Score

Vendor A 5 4 9
Vendor B 4 3 7
Vendor C 1 1 2


I hope that helps! Let me know if you have any other questions.

Ashwin Sekar

Thank you, Allison! Yes, we are eventually going to populate these criteria into an Excel sheet and weight it along with the Vendors just as how you have mentioned above. 

I agree that some of the criteria were subjective. I din't want it to be very specific because there is another section below the technical criteria that has the scores, similar to what you have mentioned (what does a 5 or 4 denote for that particular criteria). 

But I think I should rephrase some of the wordings. I am also unsure if I should include the Project Management part as we are only going to view the demo samples and won't get too much insight into their PM skills with that.  

What do you reckon?



Dave Goodman

Ashwin - some additional thoughts:

1. differentiate on the creative approach the vendor took to the design and thought process that they used

2. did any one propose an interactive video in which the learner can interact with the video?

3. look at the video demos on various devices and see if there are any differences, quality, etc.

4. look at the load time - did any vendor provide pre-loading or buffering to reduce the learner's wait time?

5. when you asked for 3 different levels of complexity, were you looking for content/visualization complexity (the wow factor) or was the complexity in the instruction design aspects of the video? This should be a selection criteria issue.

6. Asking a vendor to produce three videos for free might be a heavy lift for some vendors. Look at their approach, the types of questions or points of clarification that the vendor asked (judge them on the quality of their questions), how responsive were they, pretend that you were the learner, when you saw each video for the first time, which vendor and/or video impressed you the most (your reactive and engagement factor), etc.

7. which video concept made you become more engaged via their storytelling skills.


This discussion is closed. You can start a new discussion or contact Articulate Support.