A Kirkpatrick White Paper
April 20April 200909
The Kirkpatrick Four Levels™: A Fresh Look After 50 Years
1959 – 2009
By Jim Kirkpatrick, PhD
and Wendy Kayser Kirkpatrick
April 2009
©2009-2011. All rights reserved 2
The Kirkpatrick Four Levels™: A Fresh Look After 50 Years
A Golden Anniversary Celebration The Four Levels are turning 50 this year! November 1959 marked the first time Don Kirkpatrick published his thoughts on training evaluation with a series of four articles in the Journal of the ASTD. Don will tell you he did not call them levels, and he did not coin the phrase “Kirkpatrick Four Levels Evaluation Model”. He did, however, use the legendary four words that comprise the foremost training evaluation model today: Reaction, Learning, Behavior, and Results. On the anniversary of the four levels, we thought it would be appropriate to first honor Don and his work, and second to share with you how the four levels have both evolved and remained relevant over the last half century.
The Day of Reckoning Has Arrived When we conduct workshops on the Kirkpatrick Model all over the world, we find a surprising number of misconceptions about it. After 50 years, we figured it was time to set the record straight. Rather than attempt to clarify the many misunderstandings about the four levels, we will focus on the salient elements of the model, and show you how to leverage its power at a time when the day of reckoning has arrived for the entire learning industry. Don’s incredible vision is evident in a quote he obtained from a colleague back in 1959:
“Managers, needless to say, expect their manufacturing and sales departments to yield a good return and will go to great lengths to find out whether they have done so… likewise, training directors might be well advised to take the initiative and evaluate their programs before the day of reckoning arrives.” “The Experimental Evaluation of Management Training: Principles and Practice,” Daniel M. Goodacre III, The B.F. Goodrich Company, Personnel, May 1957
This statement represents what the ultimate intent of the four levels of evaluation was then, and is today: to show the business value and worth of training.
Donald L. Kirkpatrick, PhD
The ultimate intent of the Four Level was then, and is today:
To show the business
value and worth of training.
©2009-2011. All rights reserved 3
The Kirkpatrick Four Levels™: A Fresh Look After 50 Years
Before unveiling the new Kirkpatrick Business Partnership Model visual, we will present what we consider to be the five foundational principles. The following are the cornerstones of this model as a differentiator from others, and the keys to successful implementation of training programs. Some will likely be a surprise to you.
The Kirkpatrick® Foundational Principles 1. The end is the beginning
Most learning professionals have heard of the four levels, and many can recite them. But relatively few know how to effectively get beyond Level 2. When presenting these concepts to groups of professionals, we often refer to current evaluation practice as “smile sheets (L1), pre and posttests (L2), and hope for the best (L3 and L4).” Most of our workshop participants know exactly what we are saying. In 1993, Don wrote his first book on the Four Levels to explain the entire model (which is, and has always been, more comprehensive than four simple levels). On page 26 of Evaluating Training Programs: The Four Levels (1st Edition, Berrett-Koehler, 1993), Don writes:
“Trainers must begin with desired results and then determine what behavior is needed to accomplish them. Then trainers must determine the attitudes, knowledge, and skills that are necessary to bring about the desired behavior(s). The final challenge is to present the training program in a way that enables the participants not only to learn what they need to know but also to react favorably to the program.”
It is unfortunate that the message above has been missed by many learning professionals. For decades, practitioners have attempted to apply the four levels after a program has been developed and delivered. It is difficult, if not impossible, to create significant training value that way. All this is said to distinguish the development of the plan to build effective programs and evaluation methodology (which starts with Results, as shown to the right) from the actual data collection, which does begin with Level 1 and works forward through Level 4.
The Four Levels
Level 4: Results
To what degree targeted outcomes occur, as a result of the learning event(s) and subsequent reinforcement.
Level 3: Behavior
To what degree participants apply what they learned during training when they are back on the job.
Level 2: Learning
To what degree participants acquire the intended knowledge, skills, and attitudes based on their participation in the learning event.
Level 1: Reaction
To what degree participants react favorably to the learning event.
©2009-2011. All rights reserved 4
The Kirkpatrick Four Levels™: A Fresh Look After 50 Years
2. Return on Expectations (ROE) is the ultimate indicator of value
What normally happens when executives ask for new training? Despite what they might say, most learning professionals jump to the task by all-too-quickly retreating to their departments and commencing to design and develop suitable programs. While a cursory needs assessment may be conducted, it is rarely taken to the point that expectations of the training contribution are completely clear.
Much of the training and consulting that the Kirkpatricks do involves helping learning professionals negotiate key business stakeholder expectations. This is a process where learning professionals ask questions to clarify and refine the expectations of the key business stakeholder so that they are satisfying to the latter and realistically achievable to the former. Learning professionals then need to convert those rather generic expectations into observable, measurable success outcomes by asking the question, “What will success look like to you?” Those success indicators then become the Level 4 outcomes – the targets – to which you can sharply focus your collective efforts to accomplish return on stakeholder expectations. 3. Business partnership is necessary to bring about
positive ROE
Note in the last sentence, we wrote, “collective efforts.” Contrary to training myth and deep tradition, we do not believe that training events in and of themselves deliver positive, bottom line outcomes. Much has to happen before and after formal training. Learning professionals need help to do it, and that help needs to come from our business partners. Historically, the comfort zone of learning professionals has been Levels 1 and 2. We believe this is one of the major reasons why many learning professionals spend almost all of their time there. But the actual execution of learning programs and overall corporate strategy occurs primarily at Level 3. 2008 research by Bersin and Associates shows us that as much as 70% of employee learning occurs on-the-job in many different ways. Thus, we need to reinvent our roles by becoming experts in and deeply involved with learning that extends beyond our normal comfort level, or we run the risk of losing our positions. Therefore, not only do we need to call upon our business partners to help us identify what success will look like, but we will need a cooperative effort throughout the learning and performance processes. Before training, learning professionals need to partner with supervisors and managers to prepare participants for training. Supervisors should explain to their direct reports what they will learn in training, why it’s important, expectations for actions during and after the training eve
