## How long before we know Reimer is legit? Statistically?

The guys at Behind The Net say you need 2 or 3 seasons from a goalie before you see enough shots to get a reliable estimate of their true save percent, IIRC.

I don't have a lot of experience analyzing goalie numbers, but I do have a pretty solid background in stats, so I can tell you HOW you would be able to decide.

Nutshell version:

We can probably tell if he's a "bust" in maybe a season or two, but to know for sure that he's truly elite would probably take 3 or 4.

In statistical terms, how many shots you need depends on the size of the effect you're trying to determine. If you want to know "can the goalie stop more than 50% of the shots reliably", and your guy is regularly stopping better than 90%, that is a difference of over 40%, and you can probably see that very reliably within two or three games.

Unfortunately, NHL goalies are so tightly bunched in their skill level that what we're trying to measure is far more fine grained that that - we want to know if someone "true" save percentage is 91% vs. 94%. That means out of 100 shots taken a "good" goalie will let in only 3 less goals than a "meh" one.

Here's an analogy: If you want to see the moon at night you can look at the sky with your naked eye (you don't even have to wait till you're done giggling about the word "naked"). If you want to see a tiny star, you need a telescope to zoom in further.

Statistically it's the same thing. It 30 seconds to determine that a Reimer is better than me, takes 2 games to determine he's better than probably most NCAA goalies... but if you take any two NHL goaltenders and shoot 100 game shots at them, probably more than half the time you are going to get something like (pulling a number out my ass) 88 to 96 of them saved. The variance in this will be dominated far more by the inherent randomness of who was shooting, from where, how screened they were, how the goal was set up, blah blah blah, than by the actual skill difference of the goalies. You have to average a LOT of those "samples of 100" together in order to get an estimate that is reliable down to 1 or 2 percentage points

So if you want to know if his .93 apparent save percentage is better than a "true" save percent of, say, .88, that's a 5% gap, and you might be able to get fairly confident about that after only a season or two. This is your basic raycrap phenomenon - given enough goalies who's save percentage is truly .89, some of them will manage a whole year of really good percentage, just on chance.

If you want to know if his apparent .93 percentage is truly different than .92 (i.e., that he really is very good) then that is a difference of only 1%, and it'd probably take 3 or 4 seasons, possibly more to detect that reliably.

And now you know.

PensionPlanPuppets.com is a fan community that allows members to post their own thoughts and opinions on the Toronto Maple Leafs and hockey in general. These views and thoughts may not be shared by the editor of PensionPlanPuppets.com.

## Trending Discussions

forgot?

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

I already have a Vox Media account!

### Verify Vox Media account

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

We'll email you a reset link.

Try another email?

### Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

### Join Pension Plan Puppets

You must be a member of Pension Plan Puppets to participate.

We have our own Community Guidelines at Pension Plan Puppets. You should read them.

### Join Pension Plan Puppets

You must be a member of Pension Plan Puppets to participate.

We have our own Community Guidelines at Pension Plan Puppets. You should read them.