Hello again everyone,
Over the past month, I’ve put up a few fanposts that investigated the fates of playoff-bound teams since 2007-2008 (some that may have been informative, like this one, and others that are probably an assault on good taste and a waste of your time, like this other one). We were able to see that while having good puck-possession metrics is extremely important if you want to make the playoffs, once you make the playoffs it’s really a roll of the dice. We saw in this graph here that while eventual cup winners had higher regular season Fenwick Close than everyone else, Fenwick showed virtually no predictive value in the first four rounds of the playoffs:
Since then, Stephan Cooper wrote in this article (How good is a parity era cup winner?) about the random nature of the playoffs, the true difference in skill between various teams, and other things- but one phrase that really stuck out was that "the cup is a crapshoot, but also only a certain quality of team is allowed to get at the betting table." This is a great sentence, and really distills the reliably unpredictable nature of the playoffs down into a nice juicy morsel of wisdom. I had one issue with it though: what, if anything, is ‘the betting table’? It’s clearly not just getting into the playoffs- plenty of terrible teams make the playoffs by riding percentages and other forms of good fortune, they just don’t win cups. In that case, it might be making the playoffs with suitably high regular season possession stats. It would make sense, then, if teams with better underlying fundamentals- i.e. good puck possession teams- would be steadily more successful as the playoffs go on. But that is not quite what we see. Particularly during the first few rounds of the playoffs, the statistical relationship between regular season (or post-season) Fenwick Close and actual playoff victories is shockingly low, nearly to the point of being worthless as a regression variable. Of course, once you get to the cup finals, you reliably see teams with very good possession numbers- but in order to get to the cup finals, you have to win the conference finals, and then the round before that, and the round before that- where having good regular season Fenwick was hardly an advantage at all. So what’s going on?
To look into it more, I decided to head back into my little pile of data and address the most glaring issue with my previous analysis: I had not taken into account the individual matchups between pairs of playoff teams. After all, the playoffs are not a free-for-all gladiator combat; they’re a series of consecutive one-on-one battles between teams. Therefore, I compiled a set of data looking at a few basic metrics involving each playoff series since 2007-2008, not just all of the playoff teams since then. (No data for this year, sorry.) The first thing I did was to look at the difference in Regular Season Fenwick between the two teams going into each playoff matchup. In order to keep things simple, I classified each matchup going in as either ‘even’ or lopsided’ using an arbitrary cutoff value of 3.0 regular season Fenwick Close points or more. The logic here was that, if two teams with RS Fenwick scores of 51.2 and 52.3 play each other, (or 56 and 57, for that matter), they’re going to be essentially evenly matched as far as puck possession goes, and RS Fenwick will probably not be a very useful metric to predict the outcome of that series. On the other hand, if there is a large discrepancy (55.3 vs 50.6, for instance), that is where we would expect a significant Fenwick effect. Finally, in addition to graphing the full set of playoff matchups over those five seasons, I ran separate analyses that split the first round of the playoffs versus later playoff rounds in an effort to explain the lack of Fenwick prediction in early but not late rounds that we saw earlier.
Here is what we see:
Not surprisingly, we see that in general there are more close series than hugely lopsided ones, and among the lopsided series, you see fewer and fewer of them the greater the difference in RS Fenwick between the two teams. That makes sense. Overall, these histograms follow the shape of an F-distribution pretty nicely, consistent with the notion that Fenwick Close represents the weighted sum of a very large number of relatively small events (shot attempts over the whole regular season, between two teams, in this case). However, something a bit odd pops out: there appear to be a disproportionate number of evenly matched series in the first round of the playoffs. Now, I wasn’t sure if one should read into this or not- personally I would have guessed the opposite outcome, that the first round of the playoffs is prime time for possession monsters to pick apart the weaker teams that rode percentages into the playoffs. But maybe- maybe- the reason why RS Fenwick does a poor job at predicting early playoff series outcomes is because, relatively speaking, there are fewer series featuring actually significant differences in Fenwick scores between the two teams. But we can’t conclude that yet.
The next thing to do, of course, is to look at the outcomes of those series. Here are the same graphs as above, but transformed so that the difference in RS Fenwick scores is positive if the favored team won the series, and negative if the underdog team won:
The first thing you’ll notice is the difference between round 1 and the later rounds: there is a huge spike in the middle of Round 1 representing all of the even-Fenwick matchups in the first round, which is conspicuously absent in rounds 2 and later. That doesn’t tell us anything though, it only confirms what we saw in the previous figure. What we’re really interested in are all of the series with lopsided matchups going in (the red and green shaded bars). What we see is not terribly surprising: in lopsided matchups, the superior Fenwick team won somewhere around 2/3 of the time. However, since so many first-round series were evenly matched (23 out of 40, or 58% of them), we’re left with only 17 series that were lopsided in the first place, which is not very many to make a conclusion from. If only there was a way of expanding our sample size…
Thankfully, there is a way to do that. Instead of just looking at playoff series as individual units, we can break them down into individual games. Just as you can’t win the Stanley Cup without first winning a bunch of other playoff rounds, you can’t get your fourth win of a playoff round without having already gotten three. Additionally, breaking each series down game-by-game can also give us a good proxy measurement for how close a series is (not perfect, obviously, but I’d be willing to assert that a series that goes to seven games is probably more closely matched than one that’s over in four).
To look into this, I considered every playoff series within the data set that was ‘uneven’ going in, i.e. the difference in regular season Fenwick Close between the two teams was 3.0 or greater. I then used a type of analysis called survival curve analysis, which looks at how likely a team is to survive a series of time points (in this case, the seven games of a playoff series). This kind of analysis allows you to determine whether or not one group is statistically more likely to survive longer than another group, or whether the difference between the two could have simply happened by chance. Here is what we see:
The way to look at these graphs is to compare the green line (Fenwick-favored teams) to the red line (underdog teams) and see if there’s daylight between the two curves. If there is, then one group is statistically more likely to outlive the other group; if not, then there is no difference between them. The x-axis shows each game of a playoff series, and the y-axis shows the cumulative odds that either team has been eliminated at that point. We see something strange, and very interesting here: In all playoff rounds combined (on the left), and in rounds 2 and later (on the right), we see a separation between the curves that is both statistically significant and in the direction that we would predict: better possession teams are less likely to be eliminated than worse possession teams. Except for in round one. That is bizarre.
Now, this could be just numbers playing tricks with us- after all, I did not hypothesize this, I stumbled across it accidentally- but if it’s true, then it might mean something very interesting. It could mean that in the first round of the playoffs, two things occur simultaneously: 1) there is an overrepresentation of evenly-Fenwick matched series, which cannot be meaningfully predicted by RS Fenwick scores since they were close to begin with; and 2) even in those series where there IS a Fenwick Favorite, they barely outperform the underdogs- uniquely in the first round. Now, I have no idea why either of things would be true, nor do I believe they are linked phenomena. But let’s look back at the graph we saw a few weeks ago that prompted all of this fuss:
It could very well be that these two bizarre results we’ve uncovered- that close Fenwick battles are overrepresented early in the playoffs, and that the first round survival data is weird compared to everything else- may explain why we get such a weak relationship between RS Fenwick scores and playoff victories. If we exclude the first two playoff rounds, we get the very nice upward relationship that we were all hoping to see. So this could be why.
Should this change how we approach the playoffs, or think of playoff success? Not a ton. But it does help us redefine the betting table. If these results are for real, then it implies that the crapshoot betting table nature of the playoffs lies mostly in the first round. For whatever reason- and believe me, I’ve thought of why and I don’t know- there’s something weird about those first four-to-seven games- ask a Washington fan. But it means that perhaps ‘the betting table’ is simply the second round of the playoffs. If that is the case, then we could write an alternate version of Stephan’s crapshoot statement. Instead of saying, ‘the cup is a crapshoot, but only a certain quality of team is allowed to get at the betting table’, we could write it differently, as ‘only a certain quality of team can win the cup, it’s a crapshoot to see who even makes it to the betting table, but thankfully there is a minimum set of requirements to make it into the casino’. Definitely not as catchy. But it may reflect these new data, and explain some of the puzzling results we’ve seen before.
Looking forward to discussion.