The Curious Case Of Dan Bylsma And Michel Therrien

I don't recall who, but one of the hockey stats guys recently made a remark on Twitter that went along the lines of "Who was the last coach to be fired with a high PDO?" The idea behind that remark was the idea that coaching changes are often made on the basis of a run of bad luck rather than bad underlying statistics (St. Louis earlier this season being a good example). In today's FTB, Van Ryn's Neurologist suggested that the firing of Michel Therrien by the Pittsburgh Penguins in 2008/09 and the subsequent hiring of Dan Bylsma was a counter-example. He challenged anyone to disprove the numbers he had collected, so, being bored and on a train at the time, I decided to do that. What follows is, I would argue, an examination of the role of luck in team success and a look at what kind of effect a coaching change can have on a team's performance at even strength.

It is indeed true that when Michel Therrien was fired by the Pittsburgh Penguins in 2008/09, the team had a high PDO - 1016. Not only that, but once Dan Bylsma came aboard, the team did not have its PDO regress toward 1000 over the rest of the season, as the theory behind PDO might suggest. Instead, it rose; the Penguins' PDO during the portion of the season when Bylsma was the coach was 1028. At first glance, this appears to contradict the theory that many proponents of advanced statistics favour - that a high PDO at one point in the season will regress over time toward 1000, and that this should happen regardless of whether the coach stays the same or not. But I think if we look at the information in a bit more detail, we'll find that that isn't the case. Look at this graph:

It turns out that during Bylsma's first full season as coach of the Penguins, their PDO did in fact regress, falling below 1000. And yet, the Penguins had a considerably better record with Bylsma as coach than during Therrien's final season. Look at the records:

So what's going on here? The Penguins have a lower shooting percentage and a lower save percentage in Bylsma's first full season, but their record is far better than it was under Therrien. I think I have an explanation for that:

The effect that Bylsma has on the possession ratios is dramatic and immediate. Almost as soon as Dan Bylsma becomes coach of the Pittsburgh Penguins, they begin dominating puck possession; they go from a team that gets consistently out-shot to a team that consistently out-shoots the opposition.

So what does all this data mean? Here's my interpretation:

PDO, being luck-based, isn't a very solid indicator of how well a team is playing. It doesn't really do a very good job of reflecting how many wins the team was getting. I think the fluctuation that you see from Bylsma's first stretch with the team to his first full season further demonstrates that - the team's PDO falls drastically in 2009/10, and yet the team finished 4th in the conference, just one game behind New Jersey for 2nd in the conference and first in the division. The team's Corsi and Fenwick numbers, however, improve dramatically under Bylsma and remain consistent during the entire time period looked at here.

This is only one coaching change and two seasons worth of data for one team, but the conclusion that it leads to, I think, is that the PDO numbers you see are largely luck-driven and not an indicator of how well the team was playing, while the Corsi and Fenwick numbers are both much more reliable predictors of team success. So did the coaching change affect how well the team played? Yes. But Bylsma didn't make the team shoot more accurately and he didn't make the goalies stop a higher proportion of the shots. What he did do was put in place strategies that allowed the team to dominate puck control, overcoming the fall in the percentages, and making them one of the best teams in the NHL.