Last week, I put up a post that looked at the relationship between regular season puck possession metrics (as measured by Fenwick Close) and postseason success. (If you haven’t seen that post, read it first, otherwise this one won’t make any sense.) For anyone rooting for the predictive powers of Fenwick Close, the results were pretty underwhelming. While cup winners did show signs of better puck possession than non-cup winners, Fenwick Close was completely unable to distinguish between those who exited in the first, second, third, and fourth rounds of the playoffs. Additionally, while having a good regular season Fenwick seemed necessary to win the cup, it showed virtually no evidence of being predictive of winning the cup in any sense, as shown by this graph:
Using linear regression, we were able to generate a formula for ‘expected playoff wins’ based on regular season Fenwick, which turned out to be approximately [ PW = 5 + (RS Fenwick – 50.0)/3 ]; in other words, a team with regular season Fenwick of 50.0 could expect to win on average 5 playoff games, and each increase of 3 regular season Fenwick percentage points predicted one additional playoff win, on average. This regression equation had very poor predictive power, presumably because there are a great many things that can affect playoff success (goaltending, injuries, small number of games, etc etc). However, that doesn’t mean we should throw the equation away. What this equation allows us to do is to compare expected playoff wins based on RS Fenwick to actual playoff wins (based on reality), and then see what variables can best explain the difference between the two. In essence, we can use this difference to see what factors are spoiling our RS Fenwick/playoff wins relationship.
Using this approach, I looked at the most obvious thing I could think of that might explain why some teams go deep and others go home: goaltender save percentage. Looking at the same set of teams I looked at in my last post (all teams to make the playoffs since ’07-08, not including the current season), I looked at the regular season save percentage of each team’s starting goaltender and compared them to two things: 1) their number of playoff wins (of the team, not of the goalie), and 2) the difference between their number of playoff wins and their expected number of playoff wins based on RS Fenwick: positive numbers indicate they outperformed expectations, negative numbers meaning underperformed. Finally, I separated teams into three categories: underperforming (meaning, won three more playoff games fewer than expectations), overperforming (three or more games more than expectations) or equal, to see if regular season save percentage could explain any of these differences. Here is what the data show:
The data here show pretty convincingly that starting goaltender regular season save percentage does not really have any bearing on playoff success whatsoever. In the graph on the left, the regression line relating RS SV% and playoff wins is about as flat as Manitoba (Although unlike Manitoba, the graph contains lots of playoff teams). In the middle, we see that RS SV% does not explain any significant amount of difference between Fenwick expected versus actual playoff wins either. On the right, we can see that separating teams based on under, equal or over performance shows a hint of a trend in the right direction, but not significantly so.
That’s fine, though. We all know that playoff goaltending is what matters, right?
Now we’re talking. In the graph on the left, we can see a nice upward slope between playoff wins and playoff save percentage that certainly supports the notion that goaltending in the playoffs is important. (Note: the red dots indicate teams where the goalie with the most playoff starts in that year was someone other than their regular season starting goalie. I contemplated excluding those points, but they didn’t seem to have too much influence on the averages.) The R-squared values for both playoff wins and playoff wins relative to Fenwick expectations are now more meaningful: playoff save percentage explains 12 percent of the variance in playoff wins among all teams, and 15 percent of the variance in playoff wins relative to expected wins based on RS Fenwick. Grouping teams based on relative performance confirms this notion by showing a significant difference in playoff save % by one-way Analysis of Variance. This should hardly be a surprise, really, since goaltender save percentage in the playoffs will have a real, very direct bearing on playoff outcomes that regular season goaltending (or puck possession) will not. Speaking of hardly a surprise- check out Marc-André Fleury hanging out all by himself over there. Yup.
However, there is something interesting here worth considering further. We see here that Playoff Save % was just about equally good at predicting raw playoff wins or playoff over/underperformance. That’s good; it would be a bit surprising to see one be significantly correlated and not the other. However, we also saw earlier that regular season save% is not really predictive of playoff performance at all, whether in raw or relative-to-Fenwick terms. Obviously some goalies get hot and others get cold in the postseason, but generally the relationship between regular season and playoff sv% within goalies isn’t so weak:
There are two possible explanations for this. The first is that, although regular season and playoff save% are somewhat correlated, goalie performance during the regular season really has no bearing on the playoffs- we do not gain any explanatory power by considering regular season stats. (After all, the regression line between RS SV% and playoff wins was perfectly flat.) The second possibility is more intriguing: perhaps, playoff save% is not even what matters most: it’s the difference between regular season and postseason save percentage that has the biggest impact come playoff time. In other words, strictly in terms of predicted playoff performance: is it better to have a goalie who saves .935 in the regular season and then stays consistent in the playoffs, or one who usually saves .905 and then makes a big jump up to .930 when it counts?
To look at these possibilities, I considered all teams where the same goaltender started the majority of games in both the regular season and the playoffs (excluding 7 or so teams, many of which were the Washington Capitals) and graphed the difference between their playoff and regular season Sv% compared to playoff wins and relative-to-Fenwick-expected wins. I also separated all goaltenders into three categories, similar to before: underperforming (15 or more percentage points worse in the playoffs than in the regular season), overperforming (15 or more points better) and equal performance, to look for categorical differences between groups.
Here’s what we get:
Looks like we have our answer. The difference in RS vs. Playoff save percentage had a steeper relationship with playoff wins and with wins relative to Fenwick expectations than Playoff save percentage alone. Additionally, the r-squared values were about the same, indicating the same amount of ‘tightness’ between both relationships. Finally, separating teams by relative goalie performance shows a very nice difference in playoff performance relative to Fenwick expectations between the underachievers and the overachievers.
As another way to illustrate this, I separated all teams into two groups of three categories each: whether their goalie under, over, or equally performed to their RS Sv% in the playoffs, and whether the team under, over, or equally performed in the playoffs relative to their Fenwick-predicted wins. This graph shows a convincing shift that should be a surprise to no one, but still looks cool:
How do you like them apples?
So, what can we conclude here? Well, playoff goaltending is obviously important. But we knew that. However, playoff overachieving may be equally or more important, as least as far as the goalie is concerned. I won’t speculate too much as to reasons why this might be, and leave it for the discussion. Overall, I don’t think these graphs show anything terribly surprising, but I think they’ll still prompt some good arguments on the relative merit of puck possession vs. goaltending vs. luck when it’s all on the line. Enjoy.