If you're a regular reader of hockey blogs then you're probably familiar with Rob Vollman, or at least with some of his work. His most well-known contribution, I am duty-bound to tell you, is the excellent player usage charts that first appeared a couple of years ago. He was also one of original writers for the Hockey Prospectus web site, which over the years has played host to important articles by some of hockey's best statistical analysts such as Tom Awad and Gabriel Desjardins, in addition to everybody's favourite amateur scout, Corey Pronman. Rob has recently released a book which attempts to provide a summary of much of the current understanding of statistics in hockey, called Hockey Abstract.
The book is openly modelled after Bill James' Baseball Abstract, though as a long-lapsed baseball fan I can't tell you, faithful reader, whether or not it lives up to that billing. What I can tell you is that the book is divided largely into two parts. The first half of the book presents and attempts to answer 10 fun questions about the state of the game such as "Who is the best playmaker?" and "Who will finish first next year?" The second half of the book is something of an introduction to hockey analytics and provides an overview of important concepts like player usage (quality of competition and zone starts) and league equivalencies (translating stats from alternate leagues to the NHL). While Vollman cites the work of many other hockey writers in helping to explain his concepts (including, full disclosure, myself), the bulk of the book is original work by Rob, especially in the first half.
Stat-heads are often accused of having tunnel vision around using Corsi as the pre-eminent tool to evaluate hockey players, but Vollman wisely attempts to answer most of the questions from the first half of the book by looking at a variety of ideas and statistics and seeing if any sort of consensus can be reached from their sometimes disparate techniques. While the book does have its fair share of charts and numbers, there's very little math to be found and concepts are explained in terms that any fan of the game could understand. The most controversial of the book's opening chapters is sure to be Rob's predictions for the NHL's 2013-14 standings. While I won't spoil the chapter for you, I will say that Rob's analysis does not suggest a very happy season for the Toronto Maple Leafs.
Having finished the book, I'm at a bit of a quandary as to how to assess whether the overall structure works. On the one hand, starting the book by launching right into the most engaging material seems like a smart way to catch the reader's attention. On the other hand, the opening chapters refer quite frequently to concepts that are only explained in separate chapters that the reader will not have yet come across (often with a reference suggesting that the concept will be detailed in its own later chapter). I will explain why I'm concerned that may be a problem when I get to my section on the book's ideal audience, which gets its own paragraph later on in this review.
While the first half of the book is a lot of fun, the second half is a more clinical look at the kinds of statistics that modern hockey analysis uses, and it's in this second half of the book that my major criticisms manifest. Rob discusses a number of statistics that I personally think are of fairly limited value, and they all have one central problem: they're over-complicated. In Thinking Fast and Slow Daniel Kahneman makes a very good case that making a formula more complicated rarely adds much value past a certain point. Here's an easy non-hockey example to demonstrate that point:
Nate Silver is known for making political predictions on the basis of poll aggregation. But in addition to polling data, Silver adds a number of secret variables such as economic indicators to help make his predictions. In the 2012 U.S. presidential election Silver correctly predicted which presidential candidate would win each of the 50 states. Pretty impressive, it would seem. But Sam Wang, a neuroscientist at Princeton, correctly predicted 49 out of the 50 states and the 51.1% to 48.9% vote split using nothing other than poll aggregation. In the 2008 election Wang and Silver both tied with 49/50 states predicted as both missed Indiana. So what is gained from all the complications Silver puts into his data? Nothing.
Some of the statistics that Vollman discusses seem to show the same flaw. The first of those is Quality Starts for goalies. It's unclear to me that Quality Starts have any real advantage over SV% or even strength SV%, particularly with smaller sample sizes. The list of top goalies by Quality Starts looks very strange; does anyone believe James Reimer is a worse goalie than Steve Mason, Michael Leighton, and Vesa Toskala (among others)? Vollman specifically cites Eric Tulsky's work showing that consistency for goalies is essentially probabilistic; why then, would we care whether Quality Starts show which goalie is more consistent since "consistency" is largely an illusion? Is it more predictive of future performance than even strength SV% is? That's an important question that Hockey Abstract leaves unexplored.
This problem also shows up with two offensive statistics. The first is GVT, one of the primary statistics the Hockey Prospectus web site is known for. The list of top players by offensive GVT per game since the 2005 lockout is incredibly similar to the list of players sorted by points per game over the same span (with a slight re-ordering). The "passes" statistic that Vollman introduces in Hockey Abstract can be assessed the same way: I'm not sure what advantage it holds over simply sorting by assists per game, which produces a very similar list using simpler math.
In contrast to the sections mentioned in the previous two paragraphs, there are a couple of excellent chapters in the book's second half. The first is Vollman's work on league equivalencies. While the concept of league equivalencies is nothing new, I've never seen as in-depth an examination of them before, and it's a concept that I think should resonate with hockey fans regardless of their inclination toward statistical analysis in general. Rob's work on predicting the future output of current NHLers based on finding the closest historical matches for them is also quite good. I did this myself on a smaller scale with Joffrey Lupul when he signed his most recent contract extension, but Vollman looks at the subject in much more detail.
Having described and assessed the various sections of the book, we come to the part where I tell you whether the whole thing comes together to produce a package that's worth purchasing. I think that depends on who you are.
If you're an avid reader of hockey blogs, particularly statistics oriented ones like NHL Numbers, then there probably isn't very much here that's going to be new to you. Other than "passes" there isn't much introduced in Hockey Abstract that you won't already be intimately familiar with. If you fit into this group then whether or not the book is worth it for you probably depends on how interested you are in the ten questions asked at the beginning of the book. If you'd enjoy reading answers to questions like "Who is the luckiest team?" then I think you'll find Rob's approach interesting. If you're looking for new statistics or previously unexplored data then Hockey Abstract isn't the place you're going to find it.
If you're completely new to hockey analytics then there's a lot here for you to get into. If you're new to analytics you're most likely to be put off by the structure of Hockey Abstract, with explanations of most of the statistics used in the earlier chapters put off until later chapters. You'd likely get a lot more out of reading the second half of the book first, then flipping back to see how Rob uses those numbers to answer the ten questions he poses. It's an unconventional way to read a book, but it's probably the best way to get the most out of it if ideas like GVT or Corsi are new to you.
If you know a bit about analytics but want to know more Hockey Abstract is the book for you. Rob's style is easy to read and his explanations of many of the most important aspects of modern hockey analysis are simple and clear. If you want a better idea of how "advanced" statistics are used to evaluate teams and players, the ten questions that the book poses are a great way of introducing and explaining those concepts in a fun and easily relatable way. I can easily recommend this book to you if you fit into this thrid group.
If Hockey Abstract sounds worth a purchase to you based on what I've written, you can get a paperback copy of the book for $17.67 from Amazon; if you're ordering from Canada you'll add about $10 for shipping to that price. Alternately, you could purchase a downloadable PDF copy for $12.56. It's a bit unfortunate that there's no e-book version available, but I threw the PDF version onto my 7" tablet and found it readable, though it lacked e-book features like the ability to re-size the text.
[CORRECTION: An earlier version of this review said that Rob Vollman was the creator of Hockey Prospectus. He was in fact one of the original writers.]