Capella University Organizational Perspectives - Human Capital Management Resources - Human Capital Research

How Do Executives Really Evaluate Distance Training?

While Kirkpatrick’s “4 Levels” and other frameworks for evaluating training have been around for over 50 years, only about half of all training programs are evaluated for objective performance improvement outcomes. This has been found to be true for distance training programs, as well as face-to-face training programs.

So, how do executives really evaluate distance training?

I’m nearing the end of a year-long study of that question, and I welcome your comments, suggestions, and resources to this post.

Why is this study important? The American Society for Training and Development reported in 2006 that about $23 billion was spent on online-delivered employee training and development. In addition, nearly $37 billion more was spent on other forms of technologically-delivered training and development. So, we’re talking about a multi-billion dollar business. Yet, few of these programs are actually evaluated for performance improvement. Why?

Ed Holton and Sharon Naquin, researchers at Louisiana State University have suggested that evaluating training programs was a much more complex task than simply gathering “Four Levels” information (“I liked it,” “I learned it,” “I used it,” “It made a difference.”)

Starting with ten executives in large organizations that use distance training for a variety of purposes, I found that what Holton and Naquin asserted was true. Executives use complex criteria to decide if a distance training program was a success and whether the program should continue to receive funding or not. Also, organizations that use distance training use complex decision-making processes to determine the criteria by which distance training programs will be judged and how data will be collected to render those judgments.

I also found that some executives “skip steps” in the evaluation process. Kirkpatrick suggests that there are cause-effect relationships between “I liked it” and “I learned it” data. The executives do not use such lock-step thinking. Some executives are concerned with “I liked it” data and “It made a difference” data, but are relatively uninterested in “I learned it” or “I used it” data. In fact, “I used it” data was the least valued of any evaluation data they might receive.

My study is continuing with a second “wave” of interviews with ten more executives. This second wave of interviews will be used to confirm and extend the results that emerged from the first wave. Possibly, a theory may emerge that helps explain how executives make judgments about distance training success and failure. Final results will be reported by November, 2008, when I present at the ASTD Regional Conference in St. Paul, Minnesota. The preliminary results were presented at the ASTD International Conference and Exposition in San Diego, in June.

This entry was posted on Tuesday, August 26th, 2008 at 9:43 am and is filed under Current Topics, Featured Posts. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.