Whenever the team goes through bad times, you can hear the rattling of pikes and forks as the mob marches towards la Place du Centenaire. We want heads to roll. And Trevor Timmins is always a crowd favorite on these special occasions. Because, you know, after all these years, there can’t be any excuses left. Sure, drafting is not an exact science, but 2003! Your kids weren’t even born! If he was really good, after all these years, we’d know. Right?

The thing is, there is only one draft each year. By nature we have only a small sample to work with when we evaluate the success of a head scout. Even after 15 years the number of picks is around a hundred. When you categories, because you can’t compare top five picks with end of first round picks and 2nd round picks, etc, the sample gets thin pretty fast.

A few years ago I read a book written by mathematician Leonard Mlodinow called The drunkard's walk, how randomness rules our lives. Mlodinow gives a lot of examples of how we are wrong about things. A lot of those examples are about pro sport drafting. "Deciding just how much of an outcome is due to skill and how much to luck is not a no-brainer. Random events often come like raisins in a box of cereal - in groups, streaks and clusters. And although Fortune is fair in potentiality, she is not fair in outcomes." A Hollywood company fires one of its producers (another small sample over many years kind of job) because her last four movies were disappointing at the box office. A few years later you realise that the last three movies she worked on, and that were still in the pipeline when you fired her, turned out to be huge successes. Oups.

So should we just shut up and suffer then? What if Timmins really is bad? Should we give up all hopes to see the cup in Montreal in our lifetime because of sample size? Of course we won’t shut up. Journalists have to write articles week after week. Panelists have to talk and talk. And fans… well you know fans. So we still want to judge but can we be a little fairer about it. Can we use the numbers and try to remove some of the noise. That is what I will try to do.

Has Shania Twain would sing it: "So you consecutively drafted Fleury, Malkin, Crosby and Stall, in the top 2, and they won the cup? That don’t impress me much! So you got the rings but have you got the touch?" And how would I know if you’re actually alright? Getting the obvious star doesn’t make you a good head scout. One head scout gets to pick third in 2003 while another gets to do it in 2012. Not the same thing at all. What tells them apart is the potential available when they make the pick. So I’m thinking evaluating drafts is not so much about who picked the best player but more like who didn't let go of a huge opportunity. The scale should reward and punish. I decided to try to test that and realised that hockeydb.com’s draft tables can be copied and pasted in a spreadsheet. So I did that for drafts from 2003 (beginning for Timmins) to 2016 (later drafts are not telling enough for now) on 2021-11-11 (yes data has to be time stamped).

First equalize by position

To analyse by data you need to score things. A good way to score a player drafted is by game played and points (goals + assists). You shouldn’t be penalized for picking a goalie or a defenseman though. I decided to score like this:

• Forwards by games played + points

• Defensemen by games played + 2 * points

• Goalies by game played * a factor (3.286) to get the average similar to that of other players (wins and loss is not in that database and anyway this works fine)

Next, evaluate potential

So each pick is now given a score. But they also have a potential; that is the sum of all the scores still available at that point in the draft. For example, picking Sidney Crosby gives you a score of 2 365 (2021-11-11). But, since he is the first pick of that draft, he also comes with a huge potential, the sum of all that draft: 51 123. For the second pick Crosby is no longer available so the potential of that pick drops by 2 365 to 48 758.

Calculate a Scouting Rating

Each time a team picks, it scores points which add up. That team also accumulates the potentials associated with the pick they made. At the end you have a sum of scores and a sum of potentials for the period of your database. The Scouting Rating is total score/ total potential. I multiplied by 10 000 and rounded to get a nice 3 digit rating.

So if you pick a player who never plays in the NHL game at the beginning of the first round of a great draft year, you dropped the ball. And you get punished for it. Older mistakes punish you more because, as time passes, the potential of that draft grows (other players accumulate game played and points) while your bad pick stays at zero. In my first ranking I had Winnipeg, Atlanta, Arizona and Phoenix. The older "teams" Atlanta and Phoenix were at the bottom and the newer "teams" were at the top. That is because older picks are more punishing. So you have to have teams with similar number of picks over the same period of time to compare. I fixed that by replacing those team names to Coyotes and Winnitlanta.

This rating, I think, removes some of the noise. As a head scout, you don’t control the timing between good draft years vs when your team sucks. You don’t control either when your GM decides to trade a first round pick. The proposed rating adjusts for that.

It does not however remove the countless other noise factors that head scouts don’t control.

• You did pick one of the best players available but that player decides, three years later, that hockey is not that important to him. (Or that partying is equally important)

• You have one of the best players available on your list but the GM likes that other big guy and forces you to pick him instead.

• The coach, or the crowd, puts too much pressure too early on that gem you found and his confidence is destroyed.

• Injuries, Development, Etc.

So is that ranking fair to Trevor Timmins?

Data hockdb.com drafts 2003-2016 as of 2021-11-11 Mean: 172