Measuring Training Effectiveness
It may come as no surprise that training expenditure rose approximately 12% from 2020-2021 to approximately $923 billion*. And why that isn’t surprising, is because employees need continuous, effective training to keep up with the demands of our rapidly changing world.
But spending heaps of money on training doesn’t guarantee a force of upskilled employees. Ineffective training won’t get you the results you’re looking for. So, how do you know if your learning is effective? How do you know that you haven’t wasted money on training that didn’t work? You need to measure your training effectiveness.
What is Training Effectiveness Measurement?
Training effectiveness measurement refers to determining whether the employee training you’ve implemented has achieved the objectives it set out to achieve. Training success metrics can help answer key questions:
- Did employees learn what they were supposed to learn?
- Did they apply the new knowledge at work?
- Did they meet a business goal?
- Was this training worth the investment?
Training metrics allow you to objectively assess how effective your learning solutions are, and use this information to improve training programs to meet your business objectives. By deciding beforehand what you want the training to accomplish, you can evaluate whether it fulfilled its purpose or advanced business goals.
Once you’ve decided on your training metrics, you’ll need to collect your information and analyze your data. This is a much more streamlined process if your Learning Management System (LMS) allows you to track usage and completion of digital learning. But what if your training is delivered as Video Instructor-led Training (VILT)?
Sidekick is a virtual training assistant that helps L&D Departments solve corporate training’s two biggest challenges: learner disengagement and measuring training effectiveness. This innovative platform uses next-gen tools to engage learners in meaningful ways. As live learning sessions are run, Sidekick tracks learning and engagement activity, providing valuable data back to HR Leaders. This data can be fed into training effectiveness models to quantify training effectiveness.
How Do You Measure Training Effectiveness?
There are a number of widely accepted, research-based models that can be used to measure the effectiveness of training. We’re going to look briefly at:
- Kirkpatrick’s Four Levels of Training Evaluation
- Phillips ROI Model for Training Evaluation, and
- Kaufman’s Five Levels of Evaluation.
Most models that have been developed to measure training effectiveness use Kirkpatrick’s model as a starting point. Let’s start with Kirpatrick.
Kirkpatrick's Four Levels of Training Evaluation
If you’ve begun looking into evaluating your training or improving the way you’re currently evaluating training, you’ve probably come across the Kirkpatrick Model during your research. Kirkpatrick’s model, also the starting point of many other models, takes into account any style of training, both informal or formal, to determine aptitude based on four levels of criteria. The model allows us to measure:
Reaction – Level 1
Level 1 evaluation asks learners for their opinions of the learning experience after the training has been completed. Questions typically focus on the degree to which the experience was valuable, whether they felt engaged, and whether they felt the training was relevant. This level of evaluation alone doesn’t yield much data to help determine if learning was successful.
Learning – Level 2
Level 2 evaluation measures how effectively the information in the training was absorbed by the learners. It looks at the degree to which participants acquired the intended knowledge, skills and attitudes as a result of the training. Level 2 may be completed as a pre- and post-event evaluation, or only as a post-evaluation.
Behavior – Level 3
In this level, you’re measuring the degree to which the training has influenced the behavior of the learners, and how they are applying their new knowledge to their jobs.
Results – Level 4
Level 4 aims at determining the tangible results of the training such as: reduced cost, improved quality and efficiency, increased productivity, employee retention, increased sales and higher morale.
Phillips ROI Model for Training Evaluation
Developed by Jack Phillips and based on Kirkpatrick’s Four Levels, this model is often described as an expanded version of Kirkpatrick’s model. One of the biggest changes Phillips made was adding a fifth level of evaluation, return on investment (ROI).
Level 1 - Reaction
This level of evaluation measures the reaction of learners to the training and often collects responses using surveys and questionnaires. It answers questions like: Was the training useful or of good quality? Were the participants happy to be part of the survey?
Level 2 - Learning
Level 2 measures if learning has taken place in the training. Quizzes, pre- and post-tests, and observations from managers or co-workers can be used to collect information.
Level 3 - Application and Implementation
This level measures if the learners are using what they learned in the training and if they aren’t, why not? The Phillips Model helps you determine whether the issue lies with the application of the learning or with its implementation. This is a subtle difference but one that can prove helpful to organizations for improving training.
Level 4 - Impact
Determines whether the training had a measurable impact on the business. This helps identify whether aspects other than training were responsible for the outcomes. It allows you to see whether factors such as sales data, weather or supply issues were responsible for changes in performance metrics.
Level 5 - Return on Investment
This level uses a cost-benefit analysis to determine the value of training programs. It helps companies measure whether the money they invested in the training has produced measurable returns, and if so, what they are.
Kaufman’s Five Levels of Evaluation
Kaufman’s model, as with the previous one, is underpinned by Kirkpatrick’s Four Levels of Training Evaluation.
Level 1a: Input
Kaufman split Level 1 into two parts: Input and Process.
1a, Input includes the training materials or digital resources that are used to support the training.
Level 1b: Process
The second part of Level 1 measures the actual delivery of the learning experience.
Micro-Level Evaluation - Focus is on the Learner
At this level of evaluation, you can track job performance to evaluate whether the learning has been correctly applied in the workplace.
Level 2: Acquisition
Kaufman’s second level studies the payoffs for individuals and small groups. The ‘micro-level client’ would usually be the learner, so you would be studying whether they acquired the learning and if they used it in their job.
Level 3: Application
The third level evaluates how well participants utilize what they learned in training, in their jobs.
Macro-Level Evaluation - Focus is on the Organization
At a macro level you can measure the effects or payoffs for the broader organization.
Level 4: Organizational Payoffs
Kaufman’s fourth level measures payoffs for the organization as a whole. The ‘macro-level client’ would typically be the business or organization undertaking the evaluation. This level includes performance improvement evaluations and a cost-benefit and/or cost-consequence analysis.
Mega-Level Evaluation - Focus is on Society
This level may allow us to collect some anecdotal evidence about societal and customer impacts, but getting good, reliable data at such a high level is often costly and complicated. It’s helpful to consider the impact of learning on customers and society in some contexts, but this evaluation won’t be necessary if the business goal doesn’t include a larger societal impact.
Level 5: Societal Outcomes
Kaufman’s fifth level focused on what he termed ‘mega-level clients’. This could refer to a business’ clientele and/or to society as a whole.
Now what?
You have training programs you’ve invested in and now you want to know if they’re working.
- Choose a model that meets your measurement needs
- Determine indicators of training effectiveness
- Answer the question: What would make this training program successful? These could be new skills and knowledge, improved efficiency, or increased sales.
- Choose an appropriate method to collect data.
Commonly used training evaluation tools include questionnaires, interviews, focus groups, and observation.
- Request feedback from learners following the completion of their course.
- Analyze feedback.
- Use feedback when designing your next program to make changes, updates, and improvements that will help you achieve your metrics
Reach out to our team today and let’s chat about your team’s learning journey.
Want to LEARN MORE? Click here to subscribe for more information about bespoke training.
Share on facebook
Facebook
Share on linkedin
LinkedIn
