Cronin recently suggested that shots were a "misleading stat" and that leafs were able to control "shot quality" for and against through their style of play. This is difficult for the advanced stats community to accept for at least two reasons. First there is weak but still positive correlation between Shot Attempts and Winning.And second, much of the work done on SH% indicates that shot quality is dominated by random chance. The answer to the question can the leafs or any team meaningfully "impact" shot quality is difficult as the effects of random chance are large. Previously I looked at shot distance and it's impact to Fenwick Close, which suggested that the leafs were indeed "outchancing" the competition, but the high SH% was still concerning. Below I look at SH% and PDO to see what we can learn.
Given that let's look at the PDO data from hockeyanalysis website and see what we can glean. PDO is simply SH% (team shooting percentage) + SV% (team save percentage) with team values near 1000. PDO is often used to describe how lucky a player or team was. And generally unless shooters have an exceptional shooting skill or are backed by an exceptional goaltender then team PDO will be near 1000.
That said, in the graphic below, I plotted the Leaf PDO in 11 different 5 on 5 situations.
1. 5v5 PDO (5 on 5 PDO)
2. 5v5 HPDO (5 on 5 PDO at Home)
3. 5v5 RPDO (5 on 5 PDO on the Road)
4. 5v5 CPDO (5 on 5 Close Score PDO)
5. 5v5 TPDO (5 on 5 Tied Score PDO)
6. 5v5 +1PDO (5 on 5 PDO when leafs are up by 1 goal)
7. 5v5 -1PDO (5 on 5 PDO when leafs are down by 1 goal)
8. 5v5 +2PDO (5 on 5 PDO when the leafs are up by 2 goals)
9. 5v5 -2PDO (5 on 5 PDO when the leafs are down by 2 goals)
10 5v5 TrPDO (5 on 5 PDO when the leafs are trailing)
11 5v5 LPDO (5 on 5 PDO when the leafs are leading)
The Shot Quality Hypothesis
If the leafs (or any team) are able to control shot quality then their PDO in each of these game scenarios should have
- A higher average magnitude of PDO indicating some superior team/player skill in most scenarios and
- A tighter distribution with a lower standard of deviation (less prone to chance)
If however, the leafs (or any team) are not able to control shot quality any better then the average NHL team, then there should be large random chance driven differences in PDO in these different situations.
A Coin Toss Thought Experiment
As a thought experiment, think of this as a coin toss. If we break down a group of 500 coin tosses into sub groups that track the probability of landing a head, if you already landed a head or landed two heads or if # of head equals the # of tails etc, then we should expect randomness in these various subsets if the coin is "fair". Similarly, if the coin is unfair or the tosser has some "skill" that increases the probability of landing a head, then we should see that most of the sub-groups have a higher probability of heads (probability of heads is greater then 50% in the sub groups). And also the standard of deviation between each sub-groups will be lower. The macro summation of all these sub groups, may still have more heads then tails but the sub groups should be more random with some having more heads and other more tails if the coin is fair.
Said differently, it is unlikely (though statistically possible) that a team will be "lucky" in every situation. That is, if a team is consistently lucky (or unlucky), then this suggests (but does not prove) there maybe some skill element at play. And keep in mind this is not a rigorous proof of anything but more observation of actual data to test Cronin's assertion that leaf can control shot quality. And one other point, is that I didn't include PDO of PP or PK because those PDO are naturally further from 1000 league average due to the mismatch in players.
At any rate, if you understand the above then below are the results for all 30 NHL teams 2013 PDO in various situations. (left click the image with your mouse to enlarge)
Interpretation Of The Results
They things I'm keying on are teams with a tight standard of deviation indicating that chance is less at play. And second teams that are skewed above or below the league average of 1000 indicating that some skill element maybe at play beyond random chance. If you "buy this" thinking, then for some teams, this suggests that they make their own luck with the playing strategy they choose. For example, consider most teams like Columbus, Detroit, Edmonton, Nashville, Winnipeg who have both large variances in PDO and average PDO both above and below league average in different situations. This random behaviour is what we would expect from of a stat governed by chance.
Contrast that with Calgary or the Flyers whose PDO are below or near league average. This suggests that their PDO reflect more poor shooting and goaltending then being largely driven by random luck. Of course "everything" can conspire against Calgary so they face bad luck in all these situations but this possible but not probable. Another interesting example is a team like Penguins, Washington or the Leafs whose average PDO in each situation is above or mostly above average and whose range is tighter. This suggests (but does not prove) that there is a skill element impacting PDO as it would be unlikely for a team to be skewed positively in most of these situations.
There are some analomlies which are more confusing. For example, the Senators appear to be a below average team in terms of goaltending and shooting skills in all situations rather then those skill being at the effect of random chance. And the Sabres with the tightest PDO appear very average with some small randomness still.
Digging into the leafs a little deeper they were "unlucky" and/or their strategy performed worse when they trailed by 2 goal and more generally they performed worse (or were more unlucky) when trailing. However, their strategy performed superior or they were more lucky when they had the lead. I'll let other folks "speculate" as to how or if the leaf play strategy changed when the leafs were leading or playing catch up. Also the Penguins appear to be an even more skilled team (or luckier team) then the leafs.
Did the Leafs Control Shot quality in 2013?
This is not a rigourous proof that validates Cronin's suggestion that the leafs impacted shot quality however it does suggest what he says is accurate. The way probability works, it is possible that each of these sub groups can be skewed positively by random chance (though that is unlikely in the 11 groups but still). To make it more solid analysis we can look into a p-test to see how statistically relevant the tighter standard of deviation and higher average PDO in various situation are. That said, personally I was surprised with the results as I went about disproving that shot quality was meaningfully impacted and now wonder if there might be more to "shot quality".