Using Metrics Over Myths in Hiring

For the first time in my life I filled out the March Madness bracket. I never followed NCAA Basketball nor do I plan on following now. I did it only because of the simple idea from The Wall Street Journal to do Blindfold Brackets.

In WSJ’s Blindfold Bracket they stripped down names of the teams and substituted them with made up names. For each game they present the two teams with their made up names but their real statistics. Based only on the metrics and stripped of any affiliation or bias to actual teams you need to pick the winner.

I do not know all the teams that are playing this year.  Bias would not have been an issue for me but without the metrics presented I would have been clueless. So I filled out my first bracket. I used the following simple rule and applied it consistently for all the match-ups I was presented.

  1. I treated all the metrics as linear  interval scale from 1 to 5.
  2. I ignored the metric on Hot streak. It didn’t matter (teams will regress to the mean)
  3. Offense and Defense are my first criteria. Both got equal weight with an exception.  When a team with better offense faced a team with better defense I picked the defensive time unless Offense is 2 points better than opposing team’s defense.
  4. If both Offense and Defense are equally balanced I picked based on experience.
  5. I mostly ignored 3-point shooting because I did not bother to check prevalence of 3-point shooting in NCAA.

You can see my bracket here. On the first big day, the WSJ standings say I picked 16 of the first 20 games correctly. Let us see if I will regress to the mean.

The point is how relevant this simple example is to how we hire a candidate, choose our gurus or fund a venture. In his book, “Thinking, Fast and Slow“, Nobel laureate and Behavioral Economist, Daniel Kahneman writes how easily we succumb to irrelevant attributes in hiring people.

We rely on looks (the candidate looks the part), how she talks, etc. We invent subjective metrics like, “hustle”. We focus on most recent success or failure and ignore the history.  We hire based on interviewing skills over wherewithal to get the job done. We let the first impression and answer to the first question bias the rest of the interview process. Once we form an opinion we keep digging for selective evidence to support our case.

Kahneman recommends a metrics driven approach to hiring.

  1. Come with a set of metrics you are going to evaluate all candidates on. You will apply the same metrics and same scale to all.
  2. Treat the different metrics as additive. If you want use weighted scale but use it consistently.
  3. Make an upfront commitment that you will hire only the candidate with highest total score regardless of the intangibles and your gut feel.
  4. Design a list of questions and ways you will measure the candidates on these metrics. Score each candidate on these metrics.
  5. If you insist on image and look, define its own metric but make sure it does not contribute  more than 10% of the total.

This may not work all the time but is likely to work most of the time and you have a reproducible process compared to the one based on your gut, driven by irrelevant factors and subject to your cognitive biases.

Are you prepared to hire based on falsifiable but repeatable metrics over axiomatic myths?

And by the way, it appears my choice for NCAA championship is Kansas. Same as the most common choice of others playing Blindfold Bracket.

5 thoughts on “Using Metrics Over Myths in Hiring

  1. Reblogged this on Future Perfect and commented:
    Apropos the workshop prediction market, which had a lot of sports questions, “Iterative Path” talks about using metrics to pick NCAA March Madness brackets, and job candidates. I’ve used this for choosing houses, with metrics like rent, distance to work, school, store, church, niceness, features.


Comments are closed.