Do You Really Know What's Important?

Here's a great video of Malcolm Gladwell from the 2008 New Yorker Conference.

He discusses the "mismatch" problem of using the wrong metrics to predict success. Some really interesting ideas here. As usual, after listening to his perspective, you tend to think "Well, obviously." Yet it's quite likely a completely different perspective than you started with.

Why do we automatically assume, and simply accept, that smaller class sizes are better for students?
Why do we automatically assume, and simply accept, that testing athletes' strength, endurance, etc. through a series of drills will show their capacity to compete in their sport?
Plus a bunch of other examples.

In a way, his ideas relate to my previous post about trying to quantify the qualitative. It's not just that we've chosen the wrong metrics to examine, but that some things are simply too complex or variable (human, in other words) to even try applying any fixed metrics.

This perspective adds credibility to the shotgun approach. Perhaps we're better off trying many different things (on a small-scale, test basis, of course) to see what really works.

Instead of job interviews, for example, what if every halfway decent candidate actually tries doing the job for a couple of weeks? Sure, it would be expensive and time consuming, but is it any worse than having an ongoing staff turnover problem because you're constantly the wrong people?

When launching a new product, how do we know if it will be successful or not? Most companies apply a series of assumptions: the price is similar to what customers already pay for a comparable product... it fulfills a need that focus groups tell us customers want... it's the same shape and colour as the industry's best seller... And on and on. Yet even though we "know" all these things, an enormous number (I would even suggest the majority) of new products fail miserably.

I'm looking forward to reading Gladwell's new book when it comes out this Fall. In the meantime, this video has made me think about a few things:
1. Re-evaluate the metrics and success criteria. Do I actually know that a certain measurement is a good indicator, or is it just a long-standing assumption?
2. Re-evaluate the very "quantifiability". Can success even be measured at all using a strict, quantifiable set of criteria?
3. Test and trial. Whenever possible, see how well alternative ideas/people/products/directions work.