Why PLAYOFFS!!!1 Isn't Good Enough

By now it's fairly well known in Leafland that the Toronto Maple Leafs have not done very well this season by puck possession metrics like Fenwick or Corsi. At the moment I'm writing this Toronto sits second last in the NHL with a Fenwick close of just 43.8%, the 5th worst a team has had in the 6 seasons for which we have these statistics (since 2007-08). This has lead to a fairly significant number of arguments between those who favour statistiscal analysis, who claim that the Leafs' success is largely luck-based, and those who disagree, saying that the Leafs have earned their way into the playoffs through strong play. Those of us who favour statistical analysis have often been countered with the claim that the Leafs were making the PLAYOFFS!!!1 and other than that it doesn't matter.

I believe it does matter. I believe that NHL teams should be attempting to set in place a foundation for long term success, ideally culminating in a Stanley Cup victory. That leads to an obvious question about the effects of possession on success: how do teams with bad possession numbers do as far as qualifying for the playoffs?

I've collected data on all the teams that finished with a Fenwick Close of below 49% for every year that we have data for prior to this year, which works out to 5 seasons. A FenClose of under 49% equates very closely to being in the bottom 1/3 of the league; in the 5 seasons looked at here, 50 teams finished with a FenClose that low (though the distribution is not exactly 10 per year). I'm going to look at two things in that list: how many teams have qualified for the playoffs in a season in which they had a FenClose of under 49% and how teams made the playoffs the year following a season in which they had a FenClose that low. This should give us some idea how likely it is for a bad possession team to make the playoffs and how likely it is that they'll be successful in the future.

Of the 50 teams that have had a FenClose of under 49%, 20 have made the playoffs either that year, the subsequent year, or both, a rate of 40%. Here is the list of those teams, in reverse chronological order. It does not include Columbus, who may qualify for the playoffs this year but haven't as of yet. The numbers are all accurate as of last Sunday when I put this chart together.

Of the teams with a Fenwick Close in the bottom third of the league, only 9 made the playoffs, just 18%, which is less than one in five. 16/50 teams made the playoffs the year following a season in which their FenClose was under 49%. Of those, 9 saw their possession numbers jump above 49%, taking them out of the bottom third of the league, so their better results would be expected. Of the remaining seven, three are from this shortened season and their results may not have been sustainable over 82 games. Previous results suggest that they probably would not have. So a very small number of teams (somewhere between 4 and 7) out of 50 were able to make the playoffs following a poor possession season. This should concern Leafs fans, unless the team can significantly improve its possession numbers next season.

This next chart may prove even more concerning for Leafs fans; it's the list of teams that qualified for the playoffs in a poor possession season and the subsequent season. There are only 5 of them:

Three of them improved their possession by a fair margin the next two seasons, and that leaves the 2007-08 to 2009-10 Montreal Canadiens as the only consistently poor possession team to make the playoffs multiple seasons in a row. This list skews heavily toward the 2007-08 season, but I have no explanation for that. At any rate, this list does not bode well at all for the Leafs.

It's pretty clear that being a poor possession club makes it unlikely for a team to qualify for the playoffs. However, some people will claim that while it may be true in general that poor possession numbers make it difficult to make the playoffs, that's not necessarily true in certain situations. Might not a coach devise a system that can beat these odds and consistently make the playoffs?

The results here certainly don't support that. 12 teams appear on this list at least once. Only 2, Montreal and Anaheim, successfully made the playoffs in more than one sub-49% FenClose season. Anaheim did it twice, but once was under Randy Carlyle and once will be this year under Bruce Boudreau. The Montreal Canadiens did it three times in a row, but all of those appearances were under different coaches - Carbonneau in 2007-08, Gainey in 2008-09, and Martin in 2009-10. If there's a coaching effect there, I'm not sure how we'd see it.

Lots of people seem to believe that Randy Carlyle has implemented a system in Toronto that affects shot quality, causing the Leafs to score at an elevated rate, stop shots at an elevated rate, or both. Maybe those people will look at the fact that Anaheim made the playoffs with a FenClose of 45.7% in 2010-11 as support for this theory. What do the numbers say? Is there any evidence that his teams can consistently beat the possession numbers? This chart should answer that:

I've collected 2011-12 numbers for Carlyle's time running the team and Boudreau's as well. I've also included the numbers under Boudreau this year for comparison. One thing that's pretty clear is that Boudreau is a better coach than Carlyle is. Boudreau's Ducks have had much better possession numbers than Carlyle's did in the past couple of seasons even though they're working mostly with the same roster (I'd be willing to bet that a big chunk of the fall in FenClose this year is due to the loss of Visnovsky, who's having a great year on Long Island). The Ducks have also maintained percentages under Boudreau that are as good as anything Carlyle got, so based on the evidence here it's pretty tough to argue that Carlyle's systems were creating a unique level of shot quality.

While I didn't include these rankings in the chart, Anaheim's ES SV% over this time was roughly middle of the league most years and their SH% was in the top half but not much higher than 10th most years. The combined effect does create a PDO consistently over 1000, but not by much; roughly half a percent.

A shortened season can play havoc with percentages though, as you can see by Anaheim's numbers this year which are hugely out of line with what a similar roster has done in the past. The Leafs' 1032 is even more unsustainable. Both teams are in for a major reality check over 82 games next season if their rosters don't improve considerably.

So, to sum this all up, I think I've painted a pretty clear picture about why we should be concerned about the Leafs' fortunes moving forward. They have an almost historically low Fenwick Close, and teams with poor possession numbers very rarely make the playoffs, especially in consecutive years. There's also no good evidence that Randy Carlyle coaches a system that consistently inflates shooting or save percentages to a level that would overcome bad possession numbers on a consistent basis. And that's why those of us who favour statistical analysis are worried about the Toronto Maple Leafs.