Why ROI on Training Does Not Matter
Is Return On Investment analysis really worth the effort? Donald H Taylor does not think so, and his article cautions fellow trainers and their clients to steer clear of this troublesome issue
Wouldn’t it be wonderful if we could prove training’s value in money terms?Sure, really wonderful. So why don’t we do it?
A lack of technique isn’t stopping us. The approach of Jack Phillips at the ROI Institute extends Donald Kirkpatrick’s 1959 four levels of evaluation and is a robust ROI method. There are other techniques, too. So why aren’t they more widely used?
There are two reasons. Firstly, a practical reason: the ROI of training is both time consuming and complicated to calculate. The second reason is a question of attitude and a fundamental show stopper. We’ll come to it later.
ROI basics
First, the basics: what exactly is Return on Investment (ROI)?
It is absolutely not a demonstration of cost saving, although many studies claiming to show ROI evaluations are nothing more than that – especially in the e-learning field. Rather, it is a way of showing what you get out for what you’ve put in.
Suppose you have installed some new equipment in a factory. The ROI calculation is clear: the plant cost you something, and your output increased after the installation. The ratio of the benefit to the cost is your ROI:
Return on Investment = incremental value / investment made
This should be expressed as a return over a period, and as a percentage. So if your new piece of plant cost £10,000, and over a year produces an extra £2,000 worth of value, it will yield a ROI of 20% per annum.
Splendid isolation
People, though, aren’t machines, and making the same calculation for training proves more complex. Even for workers, whose output can be clearly measured, such as some manual workers and sales people, proving that a change in that output was only the result of training, is complicated – there is just too much else going on to easily isolate the effects of the training.
If, after training, a group of employees shows an increase in productivity, is that down to the training? Not necessarily. We need to know that the change wouldn’t have happened anyway.
We need to know, for example, how that group performed before training. Suppose they are sales people – were their sales at a flat level, or already on an upward path? If the latter, then did the training result in an increase in sales beyond the trend that already existed?
And a historical trend is not the only way an increase in performance can be caused by something other than training. Perhaps the employer made internal systems more efficient, or a competitor went bankrupt, increasing demand. To see whether these exogenous factors caused any increase in productivity, we need to divide our sales force into two groups, with only one receiving training. This second group should be as similar as possible in every way to the trained group, from their age and educational background to their locations, experience and knowledge.
If we can set up a control group like this, and be sure what the historical productivity trend was, we can be fairly sure to identify any productivity increases stemming from training. I say fairly sure, because there is still the “Hawthorne Effect” to consider, in which people increase productivity simply as a result of change or observation. In the Western Electric Hawthorne electrical plant in the 1920s, workers’ productivity increased temporarily when factory floor lighting was improved, and then increased again when it was reduced.
But if the Hawthorne effect can be accounted for (and it can). So, if we have historical data and a control group, then ROI should be calculable (if we avoid for the moment the huge subject of calculating the monetary value of the output of non-sales staff).
Yet it was only after several years of false starts that Neil Rackham, a scientifically rigorous research psychologist and founder of sales training company Huthwaite, was able to find an organisation willing to undergo a rigorous trial of whether a training programme could be shown to affect behaviour. This company was Motorola, in Canada.
The results of this trial (published in Appendix A of Rackham's 1981 book SPIN Selling) did indeed show a significant increase in sales as a result of training. Even so, Rackham warns that “there are even more tests I'd like to carry out before I'll be totally satisfied that the ideas I've described in this book will significantly improve the results of major sales”
Rackham stood to gain considerably by proclaiming the validity of the training programme he had devised. If he felt less than totally convinced by his study, this is surely a lesson for the rest of us. Understanding the effects of training to a rigorous, scientifically defendable point, is complicated; it’s very time consuming and it may not in the end be convincing to an expert in the field.
But there’s a more important reason why training ROI is just not worth the effort.
The Wrong Answer
At the beginning of this piece, I said that there was a second, fundamental reason why people don’t carry out ROI studies on training. It is this:
Apart from training professionals, nobody really cares about training ROI
If they did care, believe me, we’d be swamped with rigorous studies. The reason Rackham had trouble finding a company willing to be the guinea pig for his study was not just the complexity of setting up a representative control group, or the political difficulty of explaining to half a sales force why they weren’t getting trained. It was the constant response he received: ‘Look we’re pretty sure that the training works well enough. Skip the ROI and just get on with it'.
In other words ROI is the wrong answer to the right question.
The question (coming from managers and directors) is: `How do I know this training is valuable to the organisation?' The answer to this question, from the training department, need not be `because I can show you, through this exhaustive ROI study, the increased monetary value of productivity that will result'.
So what do they want if they don't want ROI? They want value. They want training (where training is the best solution) to make the organisation more effective. What exactly does that mean? I don't know, because I'm not in your business, but you should.
The value proposition
Every organisation has operational pinch points. Not all of them can be improved by training, but where it will help there is usually a clear value proposition attached.
If there is a series of delayed project starts because of a lack of staff with key skills, targeted training can solve that. The value proposition: faster project starts. The time saved can be reckoned into a monetary value. That isn’t rigorous enough to be an ROI study, but it’s the sort of calculation that will justify most training spends.
Is there a staff turnover problem with new recruits leaving too early? Improve and extend induction training solution for new joiners. Or better still, consider an electronic solution for briefing people before they join up. If they are going to leave anyway, train them in detail about what they will be expected to do, and have the eventual drop outs decide not to join – that’s cheaper for the organisation. Again, the cost of staff turnover far outweighs the cost of the training. The value proposition is clear and the CFO will love you.
Justifying on-going training for staff, so that they can do their jobs, is harder. This is because the benefits are further down the line. The best answer is to have already engaged in a series of exercises, like those above, that demonstrate value. More important than any ROI study, that will establish in the minds of the rest of the organisation that you understand the business issues, and are using that understanding in building your training programme.
There are plenty of good proprietary methods that give a structure to such business-value based calculations for training impact – too many to recommend a single one here. However, before beginning to justify your training, you may want to question why you are being asked for an ROI study.
Good and bad requests for ROI
There are two reasons why someone asks for an ROI justification for training. Usually, they don’t understand the complexity of a real ROI study as described above. Put them right on that, and then discuss the business problem they’re trying to solve. That will root out the value proposition and get you talking on their wavelength.
The other reason for an ROI request is more sinister – they know exactly how difficult it is. Asking for a study ensures a long period of quiet while that pesky training person goes away and tries to sort it out. Don’t stand for it. Explain that a full study will need funding, and offer an alternative, value-based approach. Given the choice, anyone serious will opt for the latter.
A business value proposition that relates directly to a perceived organisational issue is where the training department can make a real, clear difference. Identify and solve these business issues and I guarantee that you will not be asked to provide an ROI analysis of what you are doing. Instead, you will be welcomed with open arms as an important part of the team.
Reproduced by kind permission of TrainingZone www.trainingzone.co.uk. The UK's largest training community
