What are some good post-course survey questions?

User Rank Tom Kuhlmann

698 posts

Posted Thursday, June 23, 2011 at 10:12 AM  

Got this question from a blog reader who wants to get some feedback on the course effectiveness.

 

"We want to know if the course was effective for them. And entering suggestions can be accommodated also.   There are a lot of questions that could be asked, we wondered if there is a good reference on this topic that you could point me to."

 

Any ideas?


All Replies

Meg Bertapelle

172 posts

Posted Thursday, June 23, 2011 at 10:15 AM  

I find this very difficult too - choosing the RIGHT questions to ask that you can really do something about, and writing them well enough to get to the answers that really matter - I would love to hear the communities suggestions about this!


blair parkin

287 posts

Posted Thursday, June 23, 2011 at 1:37 PM  

We use some questions for our classroom courses that may be of use:

  • The goals of the training were clearly defined
  • The topics covered were relevant to the course
  • The training experience will be useful in my work

We currently use a scale of 1 to 5, Strongly Disagree to Strongly Agree, although after reading a similar post we are going to move to a scale of 4 to cut out the middle ground/gray area.

 

We also ask "Please provide comments if you rated anything 2 or less"

 

I'm definitely not an expert on this and so I too would be interested to see what others ask

 

Blair


Bob S

453 posts

Posted Thursday, June 23, 2011 at 2:14 PM  

As usual Tom, great topic...

 

In my mind it all goes back to how we want judge effectiveness right? Is it a level 1 thing where we just look at did they learn it? Or do we dive deeper into looking at if they using it, etc...

 

So questions like Blair's are good and give you a "score". Nice. But sometimes more individualized answers are what a stakeholder wants...

 

  • Name one thing you learned in the course that surprised you.
  • What two pieces of knowledge/tactics are you most likely to use in your job? or What in the training today will make you more effective in your role?
  • Give an example of how you  will apply what you learned in this course back on the job.
  • Which topic(s) did you wish their was additional or  follow-up training on?
  • How long would you estimate before the average employee is faced with a situation on the job where this training applies? (today, this week, this month, this year, never)

User Rank Mike Taylor

912 posts

Posted Saturday, June 25, 2011 at 5:08 AM  

We've had good luck with this one.

 

 

http://www.surveygizmo.com/s3/576652/Online-Course-Feedback-copy

 

It includes these star ratings(1-5)  and an open comments box along with allowing them to indicate if they would like to be contacted or not. The hope is that it is quick and easy to complete. We've had over 1000 responses total (from about 5 or 6 courses) in less than a year so it seems to be working.

 

 


Jeanette Brooks

3,630 posts

Posted Saturday, June 25, 2011 at 5:29 AM  

I'm a fan of short surveys. My favorite format is 3 questions long:

  1. The most valuable part of this course was:
  2. This course would've been more effective if:
  3. I learned something useful in this course that I'll apply on the job. (provide a Likert scale, along with space to give comments or to describe what they learned)

 


Posted Sunday, June 26, 2011 at 9:27 PM  

We had made something along the lines of Mike Taylor.

Here is the screenshot, and attached is the Excel spreadsheet.

 


Survey Questions.xls

Posted Tuesday, June 28, 2011 at 4:41 AM  

I like to use 'I statements' for participants to reflect in their learning, here is some of what I use:

 

Ifeel  FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0035003300000000000000000000000000000000000000000000000000 confident  FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0035003300000000000000000000000000000000000000000000000000 ok   FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0035003300000000000000000000000000000000000000000000000000 uneasy about taking what Ilearnt today and applying it in my role.

Ifound the facilitator  FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0034003900000000000000000000000000000000000000000000000000 involvedparticipants  FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0034003900000000000000000000000000000000000000000000000000 presentedinformation  FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0035003000000000000000000000000000000000000000000000000000 taughtthe class 

Ifound the handling of questions and comments  FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0034003900000000000000000000000000000000000000000000000000 excellent FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0034003900000000000000000000000000000000000000000000000000 good  FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0035003000000000000000000000000000000000000000000000000000 average  FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0035003100000000000000000000000000000000000000000000000000 poor.

Ihad  FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0035003000000000000000000000000000000000000000000000000000 too much  FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0035003100000000000000000000000000000000000000000000000000 adequate  FORMCHECKBOX FFFFFFFF650000001400070043006800650063006B0034003900000000000000000000000000000000000000000000000000 notenough timeto help practice my new skills

 

I Hope this helps.

-J-


Posted Tuesday, June 28, 2011 at 4:47 AM  

Well that didn't turn out to what i'd hope...

 

Take 2:

 

I feel (i)confident (ii) ok (iii) uneasy about taking what i learnt today and applying it to my role. 

I found the facilitator (i) involved participants (ii) presented information (iii) taught the class

I found the handling of my questions and comment (i) excellent (ii) good (iii) average (iv) poor

I had (i) too much (ii) adequate (iii) not enough time to embed my new skills

 

Thanks


Anita Guerin

1 posts

Posted Tuesday, June 28, 2011 at 8:30 AM  

I was the originator of this question and all of your replies are helpful.I will be applying to this to TBTs only and we want to generate survey answers that the training developer will care about and perhaps even stimulate thoughts for revisions.

My personal favorites from you for my situation are:

  • The topics covered provided enough information
  • the course content was understandable
  • The training  will be useful to my work
  • I feel confident about taking what I learned from this course and applying it to my role. 
  • Thanks all, and feep the suggestions coming


    Regina Taute

    25 posts

    Posted Tuesday, July 05, 2011 at 7:40 AM  

    We try to keep it short and embed the survey at the end of every elearning course that we develop....that way the survey and feedback become part of the course and the learners get used to completing a survey.  The questions that we landed on after much thought and debate are:

     

    What we determined was that we wouldn't ask a question if we weren't prepared to make a change to reflect the negative responses.

     

    Likert Scale:

    1. I was able to quickly find this course.

    2. This eLearning course was easy to navigate.

    3. This course was a good use if my time.

    4. The information in this course was appropriate for online delivery.

    5. This course gave me a clear understanding of the subject.

    Open Ended:

    6. Please use the space below to provide feedback on this eLearning course.


    We use the same survey for each course so that we can compare the responses. If a course falls outside of the average, we take a good hard look at it to determine how we can make it better.

     


    Posted Friday, November 18, 2011 at 2:57 PM  

    This is all great advice for Kirkpatircik's Level one, anyone have great ideas on how to check back to see if the behavior changed? (level 3?) These are ways I have heard about, but I was hoping somone had something else out there or great ways to gather the data.

     

    • Learner and peer input
    • 360 input
    • manager surveys
    • direct observation
    • compare to benchmark studies
    • use of control groups for comparison
    • pre and post surveys 

    Mary Lorenz

    27 posts

    Posted Wednesday, March 26, 2014 at 6:53 AM  

    This is what we are asking, usually between 2-4 weeks after training:

    I was be able to apply the knowledge and skills from thisclass to my job. (Likert scale)   

    Please let us know at least one specific thing you learnedin the class that you have been able to apply to your job. (please be specific)

    What percentage of your total work times requires theknowledge and skills presented in this training? (0% to 100%)

    This training has improved my job performance. (Likert scale)

    Given all of the factors, estimate how much your jobperformance has improved as a result of this training. (0% to 100%)               

    Based on your response to the prior question, estimate howmuch of the improvement is a direct result of this training. (0% to 100%)

    After training, my manager and I discussed how I would usethe learning on the job. (Likert scale)           

    I have been provided adequate resources (time, materials,equipment) to successfully apply this training to my job. (Likert scale)

    The training was a worthwhile investment in my careerdevelopment. (Likert scale)

    I would recommend this training to others. (Likertscale)


    Walter Coolman

    22 posts

    Posted Wednesday, March 26, 2014 at 8:26 AM  

    Collecting information on behavior change is a great way to determine effectiveness of the course. But i think most of the comments are missing the mark of effectiveness. You have to get effective feedback to get an accurate gauge of course effectiveness, otherwise it doesn't matter what the follow up results display. Bob S gives the best examples of getting effective feedback.

     

    Take for example, this question, "I can apply what I learned in this course to my job," graded on a basic Likert scale. This gives you nearly zero units of effectiveness. It doesn't matter if  users respond with 1/5 or a 5/5, it's user opinion and doesn't actually tell you whether they can apply the information or not. And it also makes behavior change followup ineffective. 

     

    It's hard to do, but you need to figure out how to construct a question that will give you meaningful feedback. This is often accomplished through coding or structuring the question in some way so as not to collect opined responses. Otherwise your surveys are not doing anyone any good other than you can go to the business partner and say, "See, respondents averaged a 4/5 effectiveness when asked if they can apply this to their jobs." It might look good on paper, but did they really learn anything or apply the knowledge?


    Cary Glenn

    199 posts

    Posted Wednesday, March 26, 2014 at 9:51 AM  

    "Did you learn anything that will change how you do your job or help you do your job?"

     

    This is really the only question that needs to be answered.

     

    I question the usefulness of most Level 1 (reaction/smiley) surveys. These really only measure how the person feels after the course. They don't measure if the person actually changes their behavior weeks or months later. Companies should focus more on evaluating the behavior change. Measuring behavior change can be difficult. And determining why, or why not, there was a change in behavior is even more difficult.

     

    When I was instructing in-class courses I rarely saw bad reviews. The biggest complaints I saw were that people wanted the company to buy them lunch or donuts. It was very rare that we received any constructive criticism that we could act on to improve a course. It is nice if people like the course but it is not necessary. They may not like that they have to change how they are doing their job, that isn't the instructors fault (or the instructional designer or developer).