What's The Problem With The Leafs' PK? (Part 1)

Not a PK goal but it sure looks like one.

There has been an awful lot of talk over the past couple of seasons about the Toronto Maple Leafs' abysmal penalty kill, but to date no one has been able to pin down exactly what the problem with it is. The most common refrain is "coaching", but I don't think anyone has ever actually attempted to demonstrate how coaching might be the problem. Many people have put forward suggestions, such as "they don't have active enough sticks" or "fronting is leading to too many good chances", but to the best of my knowledge no one has taken these sorts of things beyond off-hand remarks and tried to demonstrate in detail what has been wrong with the Leafs' penalty kill. I'm hoping to be able to change that.

I'm going to be dividing this into two parts, the first of which is the piece you're currently reading, and the second of which I hope to be finished later this week. The first part will be a general statistical overview, while the second part will go into more detail by breaking down game video to see if there are any trends that can be picked out. I want to be clear here that I'm not starting with any particular assumption about the root cause(s); I'm going to gather a bunch of data points and hope that in the end they come together and teach us something interesting. Follow me beyond the jump for an explanation of my sources and methodology, as well as a look at the first half of this study.

I've gathered data for this half of the study from three sources: NHL.com's statistics section was used for basic goalie numbers, Behind The Net's Team Metrics section was used for team metrics, and theninjagreg's Super Shot Search was used for shot location data on the individual goalies.

The first thing I realised when I was trying to figure out how to conduct this study was that it wasn't going to do any good to gather a bunch of numbers on just the Leafs' goalies; after all, how would I know if anything I was seeing was out of the ordinary? On the other hand, knowing how much work was going to be involved, it wasn't going to be possible to gather the kinds of data I wanted for all 30 teams in the league. Ultimately I decided that I would compare the Leafs to just one other team. But how to decide which team to compare to?

Among advanced stats proponents, I think it's generally accepted that shot volume is the primary indicator of special teams effectiveness. The actual PP and PK percentages, it's believed, are subject to too much luck over the small sample of shots that an individual team's PP and PK allows to be reliable, but shots allowed should be a good indication of how well a team is defending on the PK. It's an indication of how well a team clears the zone, how well it prevents the other team to get set up on its PP at all, and how well it prevents the other team from getting shots through. I decided to work from that theory.

Using that information, I decided to try to find a team which allowed a similar shot rate to the Leafs on the PK but a much better PK efficiency rate, and ideally whose starting goalie had an even strength save percentage somewhat equivalent to James Reimer's. I chose Reimer rather than Gustavsson because the sharp drop-off between Reimer's stellar even strength numbers and dreaful PK numbers is one of the big mysteries of the Leafs' PK this season. I found a team who fit this criteria pretty closely, as you'll see in the following tables, and that team is the Minnesota Wild.

TEAM	SA/60	SA/60 RANK	PK%	PK% RANK
Toronto	49.5	13	74.2	30
Minnesota	52.5	18	83.1	11

GOALIE	GP	ES SV%	PK SV%
James Reimer	17	0.935	0.768
Jonas Gustavsson	23	0.913	0.850
Niklas Backstrom	28	0.935	0.875
Josh Harding	19	0.934	0.902

We've got two pretty similar teams here; both are mediocre in terms of shots allowed on the penalty kill, and the even strength save percentages of their goalies and James Reimer are virtually identical. So now comes the more interesting part, where I try to see if there are any noteworthy differences between the two teams. To do this, I used the Super Shot Search to put together charts of all the shots against each of these 4 goalies on the PK this year, then looked to see if there were any differences in terms of where they were allowing shots or goals from. I wanted to see if either team was allowing more shots from what's generally considered the "scoring chance" area of the ice, so on top of each goalie's chart I overlayed a scoring chance box, like so:

And now, nearly 900 of words worth of explanation into this post, I'm finally going to give you some novel information. If you've stuck with this up until now - cool.

What I did with the shot charts after marking out the scoring chance areas is to count up how many goals and shots each goalie allowed both within the scoring chance area and outside of it. It's worth noting that the individual numbers I came up with in these counts don't precisely match NHL.com's numbers, so take them with a bit of a grain of salt, but they're reasonably close so I think they're still useful.

GOALIE	GP	GA SC	GA NSC	GA SC RATIO	SA SC	SA NSC	SA SC RATIO
Reimer	17	13	8	61.9	48	42	53.3
Gustavsson	23	13	2	86.7	42	47	47.2
Backstrom	28	16	1	94.1	75	65	53.6
Harding	19	5	0	100.0	40	28	58.8

SC = from scoring chance areas
NSC = not from scoring chance areas

A couple of things pop out at me here. The first is that goalies appear not to allow powerplay goals from outside of the scoring chance area very often. Except for James Reimer. And that's troubling, if you're a Leafs fan. At the present moment I don't have an actual explanation for why Reimer has allowed so many more goals on shots that were taken from further out than the other goalies, but I think I will in Part 2. It's possible that he's just played really badly, but it's also possible that he's been screened more frequently or faced more shots from undefended players than the other goalies have. The fact that his numbers are so drastically worse than Gustavsson's worry me a bit, though.

Another thing that's worth noting here is that, at first glance, the idea that teams face a roughly equivalent "quality" of shots seems to be borne out. Backstrom and Reimer faced a virtually identical percentage of shots from inside the scoring chance area, while Gustavsson and Harding faced numbers that were a bit off their teammate's. It's often been said that Minnesota's goalies look so good because they play a system that allows a low quality of shots against. At first glance here, though, that doesn't appear to be the case - depending on how you interpret these numbers they're roughly even with the Leafs or a little bit worse than the Leafs in terms of allowing shots from high percentage areas. From this data, Gustavsson has faced the "easiest" shots of the bunch, while Harding (who has the best PK SV%) has faced the "hardest". We are working with a limited sample here, though.

So far the data is making it look like the difference in results is on the Leafs goaltenders and not on the Leafs system, which doesn't allow all that many shots against and doesn't seem to be especially bad at allowing shots from high quality areas. The goalies certainly fare much worse by the numbers I've gathered so far. In Part 2 I'm planning on looking at the players other than the goalies in a more detail though, and I'll provide some more detail on just how difficult the shots against these goalies were based on factors other than location. Until then, I hope you've got something interesting out of this data, and please chip in with any comments or suggestions below.