Shot to the Chart. But Who's to Blame? (You Give Goalies a Bad Name)

It's no secret that the Leafs have a bad penalty kill. In fact, it's beyond bad. Before tonight's game the Leafs sit dead last at a 69.4% kill rate. What's worse is that this isn't a new problem. Keith Acton and Tim Hunter were fired because of the horrendous special teams over the past few years. Specifically, the Leafs have finished 28th, 30th, and 30th in PK since Ron Wilson arrived. Scott Gordon and Greg Gronin were supposed to solve this problem (because presumably RW had nothing to do with the PK????). But sadly, here we are...

Previously, I've tried to make the case that coaching is the problem (as opposed to goalies), but with the poor PK persisting despite the change to the coaching staff (aside from the head honcho), it starts appearing less likely that coaching is to blame. So, when Chemmy posted about our special teams, Burtch made this argument:

I think a lot of that is probably our goaltending… Reimer’s got an .808 SH SV%, and Gustavsson has a .783 SH SV%. That’s not very good (53rd and 56th in the NHL respectively – out of ALL goalies 3SA).

Indeed, those numbers aren't very good. But as with all great correlations, we have a cause-and-effect problem: Is bad goaltending the cause of the crummy PK, or is the abysmal PK causing the goalies' SV% to go down? 

While I'm not sure whether Gabe or Tom Awad or others have looked at this before, I decided to run my own analysis. After the jump: simple stats and charts. 

 

The argument that others have made is that good goalies make for good PKs, whereas bad goalies make for bad PKs. But this assumption is always made based on the SV% during the PK. This doesn't remove the cause-and-effect problem. But this is a simple enough problem to resolve, right? We can just look at EV SV% instead of SH SV%, and compare it to the PK%. Good goalies (with high EV%) should make for better PKs. Right?

Above is a chart of the EV SV% for the 47 goalies who played more than 20 games for a single team in 2010-2011, plotted against that team's PK%. While you can see that there is a slight positive trend, the R-squared is .06. In other words, a goalie's EV SV% can account for about 6% of the variance in PK performance (at least, for last season any way).

I've highlighted a few individuals to highlight this point. Both Tim Thomas and Rick DiPietro played for teams with relatively good PKs. One of them was barely an NHL goalie. The other had the best SV% in modern history. One would think that if PK performance were related to how good a goalie is, EV SV% should have a stronger relationship with PK performance. 

Accounting for Baseline Performance

OK, so whether a goalie is good at even strength has very little bearing on whether a team has a good PK or not. This doesn't support the argument that good goalies make for better PKs. This seems, in fact, to reverse the cause-effect relationship: bad PKs lead to low SH SV%. But in order to assess this relationship properly, I think it's important to account for baseline SV%. In other words, it's not enough just to look at SH SV% and compare it with a team's PK performance, since some goalies are naturally better than others; the real measure of how a goalie's performance is affected by the PK should be in their CHANGE in SV% (the difference between EV SV% and SH SV%):

The chart above displays this change in SV% (SH - EV) relative to the team PK% for each goalie. What you can see from this chart is that there's a fairly strong relationship between how a team does on the PK, and how much a goalie's SV% suffers: The crappier the PK, the bigger the hit a goalie's SV% will take while short-handed. 

**(Author's note: This relationship is even stronger if I remove a few of the goalies with less than 25 games played to about 30% variance explained or r > 5)

Importantly, because this correlation accounts for baseline performance, these effects are independent of how good or bad a goalie is. For example, even though Reimer's baseline EV SV% was high, the crappy Leafs' PK lead to a big drop off in his SH SV%, almost exactly the same amount as for Giguere. Similarly, a mediocre goalie like Mike Smith has a smaller drop-off on a team with a strong PK. 

To further bring this point home, the chart below plots each goalie's EV SV% against his drop in SV% on the PK.  You can think about the change in SV% as an index of how well the PK is performing; SH SV% should almost always be lower on the PK because the chances will be better: if there's a big drop in SV%, it's because the other team is getting better chances; if there's no change in SV%, the PK is almost performing as though there are 5 skaters on the ice.  What this chart shows is that there is essentially zero relationship between the change in SV% and his baseline (EV) SV%. Some guys (like Mike Smith) play behind a solid PK, resulting in little dropoff (or in fact a small increase) in SV%. In contrast, others (like James Reimer) play behind really porous PKs, leading to an increase in quality chances, and a bigger decrease in SV%. How much it decreases, however, is unrelated to how good the goalie is to begin with.

 

Together, I think these three graphs point to the conclusion that bad PKs lead to low SV%, not the other way around. If that's true, then there should be some reliability to these effects. Goalies that play for the same team should be similarly affected by the lousy PK, since they're playing behind the same system. If, however, the goalie instead affects the PK, there should be little correlation between how two goalies on the same team perform on the PK. 

The chart above plots the correlation in the change in SV% (SH SV% - EV SV%) for those 15 teams that had two goalies play more than 22 games last season (plus the Leafs who had three)**. What you can see from this graph is that how much a goalie's SV% drops on the PK is (weakly) correlated with the dropoff of the other goalie on the same team. Thus, it would appear the performance of the team in front of the goalie is the big factor in determining how much a goalie's SH SV% will suffer on the PK. 

**(Authors note: I removed two data points that had goalies with 20 and 22 GP, as the small sample size for these goalies made things all weird. Ideally I'll have time to update this using multiple seasons). 

Conclusions

Overall, I think these relationships show that goalies are unlikely to be the major factor that separates a good PK from a bad PK. Teams with better goalies only have marginally better PKs. Instead, the PK at the team level seems to largely determine how good (or bad) a goalie is going to do while short  handed: those goalies who play behind strong defensive systems will suffer a lot less than those who play behind porous D. While this may seem trivial, I think it's important to keep in mind when we try to start assigning blame for the Leafs' abysmal PK (*cough* Ron Wilson *cough*). Thus, a few points I think we need to keep in mind:

1. There's no substitue for good goaltending. Having a goalie who stops 93% of shots even strength is always going to be a better choice on the PK than a goalie who stops 87% of shots.
2. BUT, SV% does not seem to have a causal affect on PK%. Good goalies don't automatically make better PKs. Conversely, bad goalies don't make for bad PKs. 
3. Changes in SV% (SH - SV%) are a good proxy for how a team is doing on the PK. 
4. Better teams help their goalies keep their SH SV% up.  (If the Leafs SH SV% looks like crap, there's probably been an abundance of good PP chances for the opposition). 
5. Fixing the Leafs PK will require more than just putting a better goalie in net. Somehow, somebody has to help this team improve. Or, as Chemmy pointed out, it could make the different between glory and failure.