About a week ago I wrote a post called Lupul vs Kulemin: Who Made The Leafs Better in 2011/12? In that piece I used team averaged on-ice shooting percentages for both players for some of my calculations and this lead to considerable disagreement in the ensuing discussion. A number of people claimed that I was unfairly penalising Joffrey Lupul, whose personal and on-ice SH% were both much higher than Nikolai Kulemin's (interestingly, no one criticised me for using team average SV%, a decision that tipped the scales in Lupul's favour). I argued that Lupul's high on-ice SH% was likely due to luck and that calculations of each player's real worth needed to adjust for that. Others argued that Lupul's had better numbers in this regard because he is genuinely a more gifted offensive player. I wondered what kind of evidence we could look at to see whether there was any way to settle this more conclusively.
Thankfully, Eric T. of Broad Street Hockey and NHL Numbers has developed a method to try to answer this question. Over at NHL Numbers last month he looked at what effect some of the top players in the league had on their team-mates' shooting percentage. He found that some of the top playmakers in the league do boost their team-mates' shooting, but even for a player like Brad Richards the effect only works out to roughly 4.5 goals per year - the rest is a combination of luck and the skill of the players Brad Richards plays with. I asked Eric if he could produce the same kinds of numbers for both Kulemin and Lupul and he generously agreed to do so.I'll use Eric's own description of what he did from the article linked above (he provides more in-depth methodology there):
Four years' worth of 5-on-5 data were collected, from the 2008-09 season to the 2011-12 season. For any given player, we can collect all of the shots his linemates took when he was on the ice. We can calculate what the expected shooting percentage for that group is (see example calculation below). We know what the actual shooting percentage was, and the difference between the two would be taken as the boost that player provided to his teammates' shooting percentage.
Eric provided me with Lupul and Kulemin's numbers using this methodology. Lupul's numbers have been divided between his time with the Leafs and the Flyers (because Lupul changed jersey number part-way through his time in Anaheim, Eric's system did not easily accomodate collecting those numbers). Let's start with Lupul's effect on team-mate shooting. In the 94 games that Lupul has played with the Leafs his team-mates have had an expected shooting % of 9.0 but have actually shot at 10.6, a pretty significant jump.
|Teammate||Shots w/ Lupul||Total SH%||SH% w/Lupul|
Any player not listed here has been on the ice for 15 shots or fewer with Lupul. The effect looks pretty strong as four of the six players saw boosts to their shooting while only one saw a drop and one was neutral. The overall effect is pretty big too at a boost of 1.6%, an increase that would put Lupul's ability somewhere between Henrik Sedin (+1.2%) and Sidney Crosby (+2.1%) using Eric's numbers from the linked article. If these numbers are an accurate reflection of Lupul's ability then he's hugely undervalued as his playmaking skills are superior to Henrik Sedin, Evgeni Malkin, or Brad Richards (among many, many others). I'm not sure how true that is though for a number of reasons.
Firstly, Lupul didn't have nearly the same effect when he played in Philadelphia. In 79 games with Philadelphia in 2008-09 Lupul's team-mates had an expected SH% of 8.5 and actually shot 7.8. If Lupul is really an elite player who can raise his team-mates' SH% by a significant margin, why did they shoot worse with him in Philadelphia.
Lupul's boost to team-mate shooting appears most strongly with Phil Kessel and Dion Phaneuf. I think there's something strange here about defencemen but I'll get to that once I post Kulemin's numbers below. Because Lupul has played almost entirely with Kessel and Bozak since his arrival we have a pretty small number of forwards to look at. Bozak's shooting remains unchanged with or without Lupul. Kessel's is clearly much higher but I'm not sure how much stock to put in that.
Kessel's SH% with Lupul was 1.6% less than his career high of 15.5% (though that number includes PP time, which the numbers listed above don't) on Boston in 2008-09. Does Kessel just shoot better when he has better linemates? That explanation doesn't fly with me, since Kessel played a significant amount of time with the Savard and Lucic the previous year as well and shot just 8.9%. Shooting percent can vary so much from season to season that I think it's really hard to draw strong conclusions from samples of this size.
Bonus bizarre fact - Bozak and Kessel have had identical on-ice SH% of 12.9 playing with Joffrey Lupul while Lupul himself has shot 12.8%.
OK, on to Kulemin. In total his team-mates had an expected SH% of 8.2 and actually shot 8.2. Here's the breakdown (again including only players with more than 15 shots while Kulemin was on the ice):
|Teammate||Shots w/ Kulemin||Total SH%||SH% w/ Kulemin|
MacArthur shot exceptionally well while playing on a line with Kulemin, Grabovski shot a little bit better, and all of the other players listed here shot worse. Kulemin's effect on Grabovski and MacArthur looks stronger than Lupul's effect on Bozak and Kessel, but I have doubts about that too. Is Kulemin actually so skilled that he can increase MacArthur's SH% by 3.5% in absolute terms (or a stunning 28% in relative terms)? If MacA is on the ice with Kulemin he's also likely on the ice with Grabovski, so there may be a combined effect at work here, but when MGK have been split up MacArthur has spent time with players like Phil Kessel and Tim Connolly, not exactly offensive slouches.
This brings me to the defencemen. All of the defencemen listed here have their SH% go down while playing with Kulemin despite the big bumps he gives to his other linemates, while all of the defencemen Lupul played with other than Gunnarsson saw their SH% go up. Does Lupul have some kind of ability to generate better scoring chances for defenders than Kulemin does, and does that explain why Lupul's on-ice SH% has been so high? I can't really answer that, but again I have my doubts. Of the two, Kulemin is the one who is more likely to go to the net and create havoc out front, which is where you would think increased defender SH% would come from. Other than that I'm not really sure how a forward would be able to have a major impact on how well defencemen shoot while he's on the ice. That's not to say that it couldn't happen, but without a good explanation for it I can't put much stock in it.
And this brings me to my ultimate conclusion, which is this - I don't think there's enough data here to draw much of a conclusion from. Shooting data is just too noisy when we're dealing with samples of these sizes. I'm not saying that Lupul definitely does not impact his team-mates' shooting, but I am saying that I don't believe there is strong enough data to say with much confidence that he does, either. For that reason, I stand by my initial decision to use team-averaged team-mate SH% in my original piece. It's not necessarily true that Lupul and Kulemin affect their team-mates equally, but until we can gather more and/or better data it's the safest assumption to make (given that we have to make an assumption one way or the other). Shooting data is fundamentally noisy and it's very difficult to pick out the signal from the noise until we have a large sample of data to work with. And in this case we don't.
EDIT: Check out Eric T.'s comment and my response below. I have changed my conclusion from my initial post slightly.