How to Improve the Evaluation of Learning


We hope this finds you well, your new year off to a good start, and your resolve to make improvements and have a positive impact at work intact!

Maybe one of your New Year’s resolutions was to make improvements in the impact that your firm’s investments in learning have on its business results? Or maybe to improve your ability to evaluate those impacts? Perhaps even to elevate the strategic importance of learning? If so, we have a (belated) holiday gift for you!  Our (significantly expanded) ATD paper entitled “7 Steps to Using Analytics to Improve the Evaluation of Learning” can be downloaded here.

We hope this stimulates some fresh thinking and helps you to make progress on a problem that has long vexed our profession. Drop us a line and let us know!

(This post was sent this month via email to our monthly newsletter subscribers.  Click here if you’d like to subscribe.)

Decision science: measuring/valuing training


Laurie Bassi was recently interviewed by Human Resources IQ regarding the importance of training investments as a predictor of company performance – and how that information can best be used within your organization:

The most effective training professionals and HR professionals are those who are learning how to blend compelling quantitative analysis with the ancient art of storytelling.

Read the entire interview here.

4 principles for improving the evaluation of training


Old habits die hard.  The training profession’s focus on (Kirkpatrick) Levels 1 to 4 (or 5) evaluation is a case in point.  In some cases, people have mastered the art of using traditional training evaluation effectively.  But for the most part, training, OD and HR professionals realize that their methods of evaluating the impact of training and development have failed to produce the type of credibility (and budgets) they would like.

By adopting these four principles for training evaluation, you will create the circumstances for breakthrough results:

1.       Have noble aspirations

  • Do not undertake evaluation as a means of justifying staff or budgets
  • Do design evaluation as a tool for continuous improvement

2.       Ensure that “the perfect” doesn’t become the enemy of “the good”

  • Do not strive for an unrealistic or unobtainable standard of evidence
  • Do strive to create the standard of evidence that executives will find compelling

3.       Every evaluation should be acted upon

  • Do not undertake evaluations that fail to produce actionable insights
  • Do consistently act upon the insights that your evaluations produce

4.       Create an “organic” system for evaluating training

  • Do not undertake ROI in a vacuum
  • Do build training evaluation into a larger HR measurement agenda (create actionable business intelligence by linking records from your LMS to employee survey outcomes to your organization’s KPIs)

Nick’s Pizza


14 years ago Nick Sarillo opened Nick’s Pizza & Pub in the northwest suburbs of Chicago.  Today, his two locations take in over $7 million a year in sales.  A recent article highlights Nick’s management approach. 

Among the 10 key elements he’s used to build a unique company culture that delivers: an emphasis on learning and the importance of creating a “meaningful place” for people to work.

Sounds like an organization worthy of exploration for Good Company!

Implications of offshoring


The Offshoring of American Jobs is a great little book – a good airplane read – that was “written” by two famous economists, Jagdish Bhagwati and Alan Blinder.   (I use the term “written’’ in quotes because it is the outgrowth of a debate of sorts between Bhagwati and Blinder at a 2007 symposium held at Harvard.)

I grabbed the book as soon as it was published (in late 2009) knowing that I wanted to read whatever it is that Blinder had to say on this topic.  Not only is this topic of great interest to me, but I have extraordinary respect for Blinder’s point of view.  He is one of those rare masters in the economics profession, able to combine deep and vast scholarship with real-world insight and a common sense understanding of the world. 

As an aside, I know this from painful years as a graduate student in economics at both Cornell and Princeton.  I quickly became convinced that macroeconomists didn’t have a clue about what was going on in the real world.   (This point of view, while clearly heretical for a graduate student of economics, has now become the norm among much of the thinking public in the aftermath of the Great Recession.)  But Blinder, who I had the good fortune to have as a macroeconomics professor at Princeton, was a rare exception.  He is a truly gifted teacher-economist.

But I digress.  Here are the important insights from the book, the vast majority of them, as I expected, being attributable to Blinder:

  • In thinking about the future of offshoring, it is important to make a distinction between “personal” service jobs (those where a person must be physically present to perform the task) and “impersonal” service jobs (where physical proximity is not a necessity because the work can be done remotely through technology, with little or no degradation in quality). 
  • “Impersonal” service jobs will continue to be offshored.  Only personal service jobs are secure in high-wage nations that do not limit offshoring through legislative or regulatory actions (such as the U.S.).
  • That means that in the future, offshoring will increasingly penetrate higher and higher levels of the educational spectrum.  We see this happening already, for example, with high-end medical services (such as reading and analyzing medical lab tests and diagnostic imaging being done offshore). 
  • Blinder estimates that the number of U.S. jobs that could potentially be lost to offshoring in the foreseeable future is in the 30-40 million range. 

 What does this mean for those of us living in the U.S.?  Here are a few of the immediate implications that occur to me:

  • Even more than in the past, our future prosperity will depend on our ability to innovate, which provides the “first-mover advantage.”  It is really our only hope if we want to avoid having our “wages set in Beijing.”  [This is a quote from another famous economist and Nobel Prize winner, Joseph Stiglitz.]
  • Education will continue to be key.  Both in our school systems and in our work places, we need to focus on interpersonal skills and learning environments that foster (rather than impede) learners’ creativity.
  • Economic uncertainty will continue to be a fact of life. 

 When I combine Blinder’s analysis with my own observations of the world of work, I come to the conclusion that, as economic uncertainty continues to grow, workers will place a growing premium on certainty.  So paradoxically, the employers who get this and attempt to provide their employees with a modicum of certainty will gain a competitive advantage by doing so.  

(By this point, you may have noticed that I have been silent on the contributions that Bhagwati made to the debate.  That’s because, in my view, they were negligible.  Bhagwati, a famous trade theorist, essentially argued that offshoring is no big deal, and that it doesn’t represent a significant future disruption to the U.S. economy.  As I sat on the airplane trying to make sense of this argument, I realized this perspective comes from the classic – and in my view, ridiculous – assumption that so many economists make: that the economy is and always will be at “full employment.”  In other words, Bhagwati comes to the conclusion that he does, because he has assumed away the problem.  Otherwise the mathematics gets too messy.)

So it should be pretty evident that I concur with Blinder’s perspective.  Offshoring will reach deeper and deeper into the American wage and education spectrum, disrupting the lives of more and more people, reaching higher and higher up the economic spectrum. 

Meaningful opportunities to learn at work will become ever-more important both to the  economic security of workers and to firms’ ability to innovate.  Employers who “get” this and respond appropriately are in a much better position to compete (and survive) than those that ignore this stark reality.

Training Magazine Top 125


Training Magazine recently announced its 2010 Training “Top 125” (PWC was ranked #1 for the third year in a row). 

This is always one of our favorite lists of the year – it’s one of the very few public sources of information on how much (some) specific organizations are spending on formal learning, so it’s always fascinating to go through the fine print on the new list each year.

Unfortunately, it looks like this will be the last “Top 125,” as Nielsen Business Media announced it’s shutting down Training Magazine and its companion website.