So we're into the summer time re-hashing of what did and didn't work for various clubs in the NHL this past season. Last off-season we used this time to try and educate people about the meaning and use of a variety of "advanced" statistics - most of which really aren't that advanced and don't require you to be able to do more than calculate a percentage or count.
For those of you that missed those postings, feel free to click on the links below to get caught up.
So that brings me to where we are in THIS off-season. Frankly a lot of ink and/or blood has been spilled in the debate over the Leafs season, and much of it centred around whether or not this team was successful thanks to increased toughness, more blocked shots, defensive "systems" that pushed shots to the outside, goaltending, or just good ole luck. Lastly of course we have the oft-referenced "intangibles" which strike me as an absurd thing to reference personally, since you're basically saying you're referencing something you can't describe in any way.
Then this week we have Joffrey Lupul go out and tweet this:
contracts aren't awarded by this CORSI i am hearing all about. They are awarded for an equal value of skill and depth (at a certain position— Joffrey Lupul (@JLupul) July 7, 2013
I get fairly tired - as I'm sure many of you do - of debating the meaningfulness of various statistics in terms of team success in the NHL. A lot of us discuss the importance of Corsi, Fenwick, SV%, Goals, Hits, Blocked Shots, Fighting Majors, etc. when we want to give our personal views on what really "matters" when it comes to team building.
Unfortunately - this isn't the sort of thing that is really open to debate. Anything we track numerically is quantifiable. If someone somewhere is watching it and tracking it with numbers, we can then compare those numbers to the outcomes we're looking for: wins and points in the standings. The "intangibles" side remains in the ether in all of this - obviously if something is "intangible" then nobody can describe it meaningfully with numbers because, well, it's NOT TANGIBLE. But I digress...
So - with all of the above in mind - I decided to run correlations over the past 6 years of regular season NHL hockey between a number of various team stats and the given team's point percentage and win percentage. I then ran year over year correlations to determine the predictive power of each yearly statistic. That is to say - I determined which of these stats are repeatable with a relatively high degree of certainty.
Just to describe what a correlation IS for those of you who aren't particularly math or statistically inclined, we measure correlation using Pearson's correlation coefficient, which on the tables below is under the column R. The R value is a value between +1 and -1 that describes the linear relationship between two variables - in this case the statistic on the table and either Team Pt% or Team Win%. Values closer to +1 are said to be highly positively correlated (i.e if the team does well in that stat, they should get more points or win more). Values closer to -1 are said to be highly negatively correlated (i.e. if a team racks up a lot of that stat they're likely not doing very well). Values close to ZERO have virtually no impact on wins and losses or points in the standings.
When we want to know how MUCH of a result is explainable by a specific variable, we square the R value (the R2 column). This gives us the Coefficient of Determination, which estimates the quality of that statistic as a predictor of the correlated value (Pt% or Win%). These values will also vary between +1 and 0, and thus values closer to +1 predict a higher percentage of Pt% or Win% while values closer to 0 offer no predictive value.
First let us explore the statistics that show a high correlation to wins and points in the standings:
|Statistic||Reliability||Pt% R||Pt% R2||Win% R||Win% R2|
|5v5 Close GF%||0.079||0.808||0.653||0.773||0.597|
|5v5 Close GA60||0.050||-0.625||0.391||-0.560||0.314|
|5v5 Close FF%||0.337||0.584||0.341||0.558||0.312|
|5v5 Close CF%||0.382||0.584||0.341||0.551||0.304|
|5v5 Close SF%||0.323||0.560||0.314||0.537||0.289|
|5v5 Close GF60||0.038||0.514||0.265||0.531||0.282|
These are ranked with respect to their correlation to Win%. Everything listed here is important to a description of how a team is doing in the regular season and virtually every stat listed is a requirement for a team to be successful. The top seven statistics are all measures of goals for and/or against during the season. Obviously these would have the largest impact on wins and losses. Next come the shot metrics, all of which are reflective of factors that make a significant difference on the ice. Lastly in the middle there - you'll notice 5v5 PDO - which is just the sum of a team's 5v5 SH% and SV%. This is yet another meaningful and important way of tracking a team's success in the regular season.
So we have a collection of the most relevant team metrics in hockey for a single year - but they are NOT all reliable in the long run over multiple years. How can we tell this? Look at the Reliability column right beside the statistic. Those values represent year over year R^2 values for each of the stats over the 6 years of data available. The higher the numbers the more repeatable a given statistic is at the team level year to year.
Look carefully at the 5v5 Close metrics. 5v5 Close Corsi For % is quite highly repeatable - it's the most reliable metric on this list year over year. It is also highly informative of a team's likelihood of winning games. If you want a stat that tells you if your team is doing well, that is likely to mean anything in the future, this is probably the best statistic you can make use of.
Now look carefully at some of the other "important" yearly statistics. 5v5 PDO and the GF60 and GA60 stats. Notice how low their reliability scores are? That's because there's a large amount of variation in how much teams score or how many goals they allow year over year. This is because SH% and SV% are NOT repeatable, reliable statistics at the team level. Yes one player might be consistently good or consistently bad, but the randomness of all of his team-mates (the guy on the hot streak - the guy in a funk) has a way of balancing all of this out over the course of a season.
So while PDO and Goals For and Against mean a LOT in the standings, they aren't something you can rely upon to remain the same in the future.
Ok so those are the stats that correlate significantly to a team's winning or losing in a given season (the ones that really make a difference and matter). Let's have a look at the middling stats that do make a difference, but do not correlate quite so highly. These are the stats that may have more to do with the style of game a team relies upon, that DO affect winning and losing - but they don't correlate as highly to team Win% or Point% because it is possible for teams to win despite being less strong in one specific area.
|Statistic||Reliability||Pt% R||Pt% R2||Win% R||Win% R2|
|5v5 Close PDO||0.002||0.514||0.264||0.489||0.239|
|5v5 Close FA60||0.337||-0.477||0.227||-0.458||0.210|
|5v5 Close CA60||0.443||-0.468||0.219||-0.447||0.200|
|5v5 Close SF60||0.257||0.459||0.211||0.436||0.190|
|5v5 Close FF60||0.258||0.457||0.209||0.434||0.188|
|5v5 Close SA60||0.284||-0.441||0.195||-0.427||0.183|
|5v5 Close CF60||0.349||0.411||0.169||0.382||0.146|
|5v5 Close Sv%||0.006||0.415||0.172||0.350||0.123|
|5v5 Close DZFO%||0.361||-0.361||0.130||-0.343||0.118|
|5v5 Close OZFO%||0.297||0.371||0.138||0.339||0.115|
What you'll notice in this section are the Face Off statistics, Team 5v5 Sv%, and the component rates that make up Corsi and Fenwick. These are all relevant but they make a difference at the margins. Some aspects are VERY repeatable, for instance 5v5 CA60 is the most repeatable of the 90 statistics I examined. Meanwhile others - that we classically think of as extremely important - PP% and PK% are very unreliable year to year, and don't have a major correlation to Win% or Pt% anyway. 5v5 play is obviously far more of a factor to winning than the PP or PK. Not that either aspect doesn't matter, it's just possible for teams to win games without those two factors working in their favour.
Face Offs are also noticeably a fairly reliable feature of a team... but they're getting down in to the lower end of meaning with respect to correlation to winning or pt%. So yes - it's nice that we brought Tyler Bozak back to win draws, but his Face Off prowess is unlikely to make us a better team if he makes the team worse at 5 on 5 when he ISN'T taking a draw.
Lastly - let's look at the statistics with the LOWEST correlations to Win% and Pt%. These statistics show virtually no correlation to winning. That means that teams can win and lose whether or not they rate highly or not in these statistics.
|Statistic||Reliability||Pt% R||Pt% R2||Win% R||Win% R2|
|5v5 Close Sh%||0.007||0.267||0.071||0.303||0.092|
|5v5 Close NZFO%||0.358||0.050||0.003||0.063||0.004|
You'll notice that team 5v5 Sh% is near the top of this list. It also has very low reliability. Then you'll see all of the penalty, hit, and RTSS stats. Virtually none of these matter to teams winning and losing. Teams win with an edge, or win without one. They also lose with an edge and lose without one. Being big and tough is NOT a cure all to a losing franchise... getting better at puck possession and spending more time in the other team's end is.
You'll also see that 3 of the 4 lowest ranked stats are Road, Home and Total hits. None of them matter particularly... but it's interesting to see how reliable Home hits are while Road hits are virtually non-repeatable. Obviously this indicates serious bias by score keepers around the NHL. Similarly Home Giveaways and Road Giveaways diverge enormously in terms of their reliability year over year.
So what does this all mean? Well - it looks like the Leafs are fixated on building a team that does certain things reliably. They want to win Face Offs, and win Fights, and lead the league in hits and blocked shots. Bring that lunch pail along and get to work etc. Unfortunately none of those things are remotely likely to change their actual play in the over-all standings. The reason the Leafs won last year was leading the league in 5v5 Team Sh%, and having a vastly improved 5v5 Sv%. They led the NHL in PDO. But as we can see above, all of those stats are amazingly unreliable and thus not likely to repeat next year.
Unfortunately - when it comes to shot differential metrics - the Leafs were god awful last season, and finished close to the bottom of the rankings. They bought out one of their best possession players in Mikhail Grabovski and let another excellent piece walk in the form of Clarke MacArthur, but decided to keep a stellar Face Off man in Tyler Bozak (who incidentally looks horrible by any puck possession metric). They've replaced some of what they lost by overpaying David Clarkson - who does all that other stuff that doesn't really matter too. They also traded for Dave Bolland who may or may not be effective in terms of possession.
I don't personally see a huge improvement on the horizon for the club, and think a serious regression is due next year in many of the un-reliable statistics the Leafs excelled in this year. If it doesn't happen - then that's wonderful for Randy Carlyle and Dave Nonis, but in the longer term they need to improve their fundamentals if they hope to win consistently.