Shaping Learning Metrics with Outcome Thinking - First...

How did we used to fight fires when there were no fire trucks, no high powered hoses, and no hydrants? Quite simply we threw water on whatever was burning. We used buckets or whatever else would hold water. And it usually involved a great deal of running to the scene of the fire with buckets full of water, and then back to the source of the water with an empty bucket to refill. If there were others to assist with the fight, they were organized into something called a bucket brigade and handed off buckets one to the other up the line to douse the flames. Granted, that process was not very effective, but it was the technology of the age. How did folks in those days measure success? Simple - the fire was extinguished, right?

Funny how folks back then had a better handle on measuring successful performance than we do today.

Today, we would be measuring how quickly one could run with a bucket full of water. Some may even develop specific competencies around that effort. We’d build elaborate simulations to train people how to run. We’d have short little learning objects on how to dip a bucket into a horse trough and fill it with water. We’d buy a million dollars worth of Learning System to track who had completed their bucket certification. We’d have leadership spewing corporate values that espouse that our fire fighters are our greatest asset, and HR would use behavioral interview guides to identify talent with attributes of those fleet of foot and with a powerful grip. We’d have contests and recognition programs geared to reward those top performers who were quickest with a bucket full of water. And still we would not be able to offer a reasonably accurate assessment of how effective our training efforts really were.

Why not? My goodness, we have competencies, and we even have competency reviews. We’re adding a Performance Management module to our learning system to link competencies to training. How could we not manage to report effectiveness? Why? Because we’re wrapped around the metric axel. We’re being beaten into submission to report on “better, cheaper, and faster”. Who cares if you can run like hell with a bucket full of water if you spill ¾ of it enroute to the fire. Aha...is that a hint? Should we be measuring more than speed?

Actually, is not a hint, it’s a trick question.

While the efficiency of carrying water should be considered along with urgency, and those are indeed metrics that matter, the primary metric of success is so very simple – Did the freaking fire get put out?

That is the primary outcome that matters.

Certainly there are variables of how quickly did it get put out, but those include extenuating factors that had nothing to do with the human performance contribution to extinguishing the blaze; like the availability of people, buckets and water just to name a few.

I use the metaphor of fighting fire because that’s what we seem to do every day in the course of our jobs. The training organization is in many ways identical to the fire department. We’re conditioned to fight fires. We’re conditioned to answer the bell. When it goes off, we’re down the fire pole, onto the truck, drive like hell to get to the fire, and spray water on it until it goes out. And if it doesn’t go out, we continue to throw water on it until it does – or we run out of water.

Not every fire can be put out with water, and not every "fire" we fight in the course of our business day is solved by training. It’s no wonder we can’t deliver meaningful metrics when we’re conditioned to resolve performance challenges with training. But it’s not Training’s fault. Stakeholders are conditioned to think the same way. They are the one’s requesting the bucket brigade from training to come throw some training at a particular problem burning in their unit.

So we throw it, and we throw it spectacularly. We average 4.9s out of 5.0 on our Level One course evaluations, and our participants consistently score above the passing threshold on the expertly designed and validated Level Two test instruments. And we can report how many butts in seats, and how many travel dollars we saved by moving learning on-line, and cost per student day is trending downward. Ain’t life grand?

But did the freaking fire get put out? How quickly? How much collateral damage? What was the source (root cause) of the fire? What can be done to prevent another fire? How well did we do fighting it? What could we have done differently to improve the outcome?

There are many more questions such as these that must be examined and asked. But –
these questions do not replace the importance of all those things I mentioned earlier like how quickly can one run with a bucket. These new questions simply precede them. They serve as benchmarks around what success looks like. They are the metrics that serve as proof of success. But they serve another purpose – they anchor critical analysis and thinking around one thing – successful business outcomes.

If the training department has not taken the time to investigate the root cause(s) of performance challenges with the stakeholder and key performers involved, and they have not clearly identified what a successful outcome should be – then they may very well be fighting a fire with water that instead should be smothered. Training; no matter how well designed and delivered and tested and tracked may only be a small part of the solution. And there is no way of knowing that in advance without accurate root cause analysis and developing clarity around what successful resolution will generate in terms of measurable outcomes.

In fact, identifying deficient business outcomes serve as the starting point for performance consulting competencies to be applied by the training organization with select members of the stakeholder’s team. Not a single storyboard should be developed – Not a single test question should be formulated – Not a single minute of training should be delivered until learning metrics have been defined in terms of desired business outcomes.

Evaluation planning is a new training competency where metrics intended to demonstrate the impact of learning are determined. This can only happen if deficient outcomes have been accurately identified and the contributing performance factors analyzed to determine the root cause of the shortfall. It’s only at the root cause level that specific knowledge and skills can be identified that are contributing to the shortfall. And what we miss by not doing this due diligence is recognizing how many other things are contributing to the shortfall besides deficient knowledge and skills. That effort tells us how big the fire really is, and whether spraying water on it will ever put it out. Without that effort, what the fire doesn’t destroy a thousand gallons of water will.

Without the due diligence of root cause analysis and accurate determination of success metrics we will have little, if any, success in reporting what every leader seeks – How successful was the training?

RoOI prevents ROI – Return on Our Ignorance of the root causes of performance challenges prevents post-training determination of Level Three & Four metrics that contribute to defining Return on Investment.

Josh Bersin just gave a webinar presentation on September 6th titled “Increasing the Value of the Learning Organization” where he went a step further and said that “Kirkpatrick’s model is outdated”. I tend to agree. What’s not being said is that Jack Phillip’s ROI model is the Holy Grail either. In fact, Bersin believes, and I agree, that acquiring ROI is often a futile attempt that wastes valuable time and resources when it comes to measuring the impact of learning. Do not misunderstand, ROI is a valid metric. My only point is that there is a time and place for ROI, and it should not be a standard target of measure. What really does matter always goes back to the simple question – Did the freaking fire get put out? And the rest of those questions I mentioned about origin, prevention, efficiency, and such.

Certainly evaluating success at Levels Three & Four provide valuable feedback, and I will continue to espouse and pursue them only where acquisition is both actionable and relevant to continued successful performance, not just a check box effort to measure for measurement’s sake. Again, that can only happen if there is lock-step alignment with desired business outcomes, the firefighting tools available, and those who called in the alarm.

How can this happen?

Let HPO help you align your learning organization with the processes and competencies necessary to identify root causes before the alarm sounds and you’re on your pre-determined path to fight a fire. Why spend valuable resources to fight a fire with water when a modified compensation plan could smother it more effectively and efficiently? Why not pursue some proactive fire prevention with critical competencies that enable effective performance consulting investigation when someone first smells smoke?

Contact HPO today for an initial consultation!

Gary G. Wise
Founder/Principle
Human Performance Outfitters, LLC.
(317) 437-2555
g.wise@humanperformanceoutfitters.com