eLearning Effectiveness – Measuring Options

Designing Digitally


eLearning Measurement

There is a growing dilemma faced by those of us in the learning and development industry. With eLearning playing an increasingly important role in the training of employees, what is the best way to assess the effectiveness of our efforts? Is calculating the ROI of eLearning the best method, and if so is it even possible to do? Or is it better to measure eLearning using an objective tool such as Kirkpatrick’s four-level training evaluation model?

This is a debate that has supporters on both sides, and valid arguments for each point. Let’s start by saying we can all likely agree that a well-trained workforce is essential to the success of an enterprise. However, training must be done within a reasonable cost structure, and must have the intended impact on the employees it’s meant to serve. This is the return on investment argument put forth by those in charge of budget decisions. At the same time, it’s very difficult to quantify things like the long-term benefits of eLearning, not to mention the opportunity costs of not doing it, or of doing it poorly. This is the argument generally supported by eLearning specialists.

So who’s right?


A closer look

In 1959 Donald Kirkpatrick published his Four Level Evaluation Model in the Journal of the American Society of Training Directors. But more than half a century after it was first conceived, is Kirkpatrick model still relevant? And even more to the point, is it the best way to evaluate the effectiveness of eLearning? At the time his model was considered primarily academic, and wasn’t truly popularized until the mid-nineties when his book, Evaluating Training Programs was published. Interestingly, this was also the beginning of wide-spread use of eLearning within the corporate training environment.

Kirkpatrick’s four-level model for evaluating the learning process is both simple and comprehensive. It consists of a series of observed steps:

  1. Reaction – How did the participants react to the training?
  2. Learning – What skills and knowledge did they gain from the training?
  3. Behavior – In what ways was their behavior (job performance) changed by the training?
  4. Results – What tangible or measurable results were achieved by the training?

It’s this last step that holds the most promise for evaluating the effectiveness of eLearning. Potential measurable outcomes include increased productivity, higher customer satisfaction ratings, more sales, improved employee retention, etc.  However, it’s still difficult to quantify these tangible effects as a direct result of training. So now let’s take a look at how we might evaluate eLearning from an ROI perspective.

Can we use traditional ROI to measure eLearning effectiveness?

In a basic sense ROI is a measure of business benefit that evaluates the performance of an investment based on dividing the results by the input cost. In the case of evaluating eLearning effectiveness, the investment is the commitment to training the workforce, the input cost is what it takes to develop and deliver eLearning courses, and the results are how well-trained the workforce is after participating in the training. But as you might see, there are a few problems with this calculation.

The first problem is in figuring the results. The benefits of eLearning once it is developed and delivered are ongoing, making the measurement of its results an ever-increasing value. The second problem is in how to evaluate improvements in business performance and assign a percentage of these results to the delivery of eLearning. And a third, but by no means final, problem with the use of ROI in evaluating eLearning effectiveness is the opportunity cost. If no eLearning is done or if it is done haphazardly, what impact will it have on the business?

Using ROI to measure a business investment is a purely financial calculation. Using it to measure the effectiveness of eLearning introduces variables that can’t be readily quantified, and therefore leave out a substantial part of the benefit of eLearning. The debate over this subject will surely continue, but let’s leave it with this final thought. While the expense of traditional training is measured one employee at a time, the cost of eLearning is spread over the entire workforce.

Here at Designing Digitally, we’re experts at delivering eLearning that is effective, affordable, and delivers the best value to your workforce. We know through experience the positive impact eLearning can have on employee development when it’s done effectively. Trust us to deliver employee training that has a positive impact on your workforce now and in the future. Call us to request more information about how our eLearning design team can help your organization.