The notion of the Kirkpatrick evaluation model as urban myth is perhaps a mental stretch, so ingrained is many people’s belief in it, but allow me to elaborate. Every now and then, a foolish notion takes such a firm grip on the public consciousness that no amount of hard evidence to the contrary can persuade its believers to put aside their convictions and embrace what is frequently an unpalatable or less interesting truth. Some such notions emanate from the ‘supernatural’ school and demand high levels of blind faith from their adherents. The absence of anything remotely evidential in the stories that surround faith-based urban myths presents no problem to their originators who, through their powers of persuasion and the vulnerabilities of their audience, succeed in recruiting armies of supporters to their cause. The uneventful passing once more of Harold Camping’s revised deadline for the end of the world on 21st October 2011 was unlikely to persuade his followers that The End Times is a put-up job any more than readers of horoscopes will cancel their subscriptions just because none of the foretold events actually happen. Faith is like bindweed – once established, tough to kill.
Some urban myths are lightweight confections whipped up by pranksters seeking nothing more than the inner satisfaction of knowing that they have duped the gullible. The Kidney Heist Hoax is a masterpiece of the genre. In its frequent beery re-telling, the narrative gathers both mass and momentum like a snowball rolling down a ski slope. Each storyteller attaches his or her own embellishments and invigorates the story by making it their own; or at least “a friend of friend’s”. These myths derive their currency from the frequency with which they are told and the conviction of the teller, no matter how implausible the story itself may be. It would seem that for many, a myth repeated often enough will assume the authority of truth.
Sometimes the zeitgeist is seized by a story, the source of which is a real event with real facts but for which the offered explanations are deemed unsatisfactory. Unlike the supernaturalists, conspiracy theorists flaunt their carefully selected, redacted and presented facts to create a compelling but not quite conclusive case in which any loose ends or apparent contradictions can be attributed to unseen malign forces. The power of the conspiracy theory lies in its gritty faux-realism wedded to an underlying mistrust of the agencies and institutions that control our lives. Even though we kind of know that Governments like all organisations are populated by well-intentioned but flawed human beings who are no more capable of hitting a deadline, managing a budget or keeping a secret than the rest of us, our collective paranoia compels us to believe that ‘They’ are capable of intricate conspiracies designed to hide from us the ‘real’ truth about UFOs, JFK’s assassination, Climate Change and the like.
The fact is that while urban myths are palpably untrue, that doesn’t stop them being believed. On which bombshell, I shall come to my point. At the risk of appearing just a little parochial, the myth that exercises me most at the moment is the widely propagated belief that Kirkpatrick’s model for the evaluation of training is a reliable tool for measuring the effectiveness of training. Not perhaps the issue at the forefront of the global news agenda, what with the Arab Spring and the collapse of the European financial system and all that, but we are wasting tens of millions if not billions of pounds every year on training that doesn’t improve workplace performance and we’re only getting away with it because we don’t routinely capture the numbers that would expose our wastefulness.
Like those who wait in vain for The Rapture, successive generations of trainers around the world have placed their faith in Kirkpatrick’s ‘levels’ and don’t bother to question whether there is any point in asking participants whether they have enjoyed the training or whether they learned anything or whether their behaviour changed. They just do it… because that’s what happens at the end of a training course. According to the American Society for Training and Development, 92% of training practitioners in the USA undertake a Level 1 ‘reactionnaire’ evaluation after a training course. Why, when in the same survey 15% of respondents accepted that such an evaluation offered little or no value? All sensible commentators accept that Level 1 evaluation provides absolutely no information that could be used to assess whether the training has had any positive impact on workplace performance. And yet it is the mainstay of training evaluation effort around the world. The unforeseen consequence of this misplaced faith, is that trainers wittingly or otherwise focus their efforts on getting ‘good Level 1 scores’ because that is the lingua franca of the world of training and very few people outside the rarefied atmosphere of conferences talk about anything else. It is frankly depressing to observe the obvious joy that otherwise capable trainers derive from being awarded ‘straight 5s’ by their participants when even a momentary pause to think would reveal that it means nothing.
I’m not normally susceptible to conspiracy theories but I do wonder how it came to be that a global training industry that invests so much time and energy measuring the wrong things whilst steadfastly ignoring the painful truths that would be evident if we measured the right things. Why aren’t uncomfortable questions being asked? Where are the screaming headlines? Why aren’t those responsible being held to account? Could it be that we are all – trainers, buyers, learners, line managers and senior management – involved in a giant conspiracy to maintain a shabby status quo because a more rigorous alternative would involve expense, time, effort, new skills and for some, failure. If that is the case, then all that is necessary for this disgraceful state of affairs to prevail is that we continue to ignore the inconvenient facts and do nothing.
Incidentally, the quote that gives this piece its title is attributed to the great Irish statesman and philosopher Edmund Burke. In fact he said no such thing. It’s just another urban myth.