Talent distribution – Goaltending (Part IV)

This is the 4th article in the talent distribution series. You can find the other articles here: Part 1, Part 2 and Part 3.

This piece will focus on goaltending.

What is rink bias?

You can’t really discuss goaltender statistics without also mentioning rink bias. Very basically, you can say that rink bias is shot tracking differences from arena to arena. In some arenas shot locations are tracked too close to the net, while in other arenas shot locations are tracked too far from the net.

The problem is that shot tracking is done by the home team, and even if these trackers do their very best, there will always be small differences between them. When the same person tracks all home games of a specific team, these differences will accumulate based on home arena.

In a perfect world all shot tracking were automated, so we would have precise shot location data. In a good world all shot tracking were done remotely by a neutral company, so that all trackers would get the same training to minimize differences. And most importantly, trackers could be randomized so that even if there still were differences these wouldn’t accumulate to specific teams.

However, in the current world, shot tracking is done by the home team, and unfortunately that leaves us with biased shot data.

Rink biases will mostly affect home data, since on the road rink biases will mostly even out. To illustrate this, I’ve looked at cumulated GSAx home and away for a few teams. I’ve used GSAx data from evolving-hockey, because it’s not adjusted for rink bias.

Here’s evolving-hockey’s cumulative GSAx home and away for NYR and MIN:

We see quite extreme differences between home GSAx and away GSAx. In New York, home GSAx is larger than away GSAx, meaning shots are tracked too close to the net and goaltenders are overrated. The opposite is the case in Minnesota.

For comparison, here’s the cumulative GSAx using my model:

Now we see much closer home and away performances. Obviously, these rink bias adjustments are based on estimations, so they aren’t perfect. But, I think they give a more realistic indication of goaltender performances.

A while back I wrote this article on rink bias. I think the overall takes still hold.

You can download a list of my rink bias adjustments here. The adjustment is given as a constant, that you can multiply with the unadjusted xG values. If the constant is higher than 1, then the unadjusted xG values are too low (think Minnesota). If the constant is lower than 1, then the unadjusted xG values are too high (think NY Rangers).

My GSAx model

I won’t go into great detail, but my GSAx model is slightly different from others out there. Normally, a GSAx is based on fenwicks against, which credits the goaltender for shot misses. I’m not quite sure that should be the case, and I wrote an article about it.

Instead, my model is based solely on shots against. This way it only credits goaltenders with actual saves. Other than that, I think my model is similar to other models out there.

The connection between GSAx and dFSv%

When we’re talking about modern day goalie stats, it’s typically GSAx (goals saved above expected) and dFSv% (delta fenwick save percentage) we’re talking about.

GSAx = xGA – GA

dFSv% = FSv% – xFSv%

So, GSAx is how many goals the goaltender has saved above expectation and dFSv% is how much better the fenwick save percentage has been compared to the expected save percentage However, not everyone realizes that dFSv% is also the same as GSAx per fenwick against.

dFSv% = FSv% – xFSv% = (1-GA/FA) – (1-xGA/FA) = xGA/FA – GA/FA = (xGA-GA)/FA = GSAx/FA

This is worth noting going forward. You can say that GSAx is the “totals” version and dFSv% (or GSAx/FA) is the “rates” version. In other words, GSAx is to GAR what dFSv% is to GAR per 60. Only with goaltenders we use fenwicks against instead of ice time.

When we look at talent it’s better to look at the “rates” version (dFSv%), because it’s independent of workload/ice time.

As mentioned earlier, my model is based on shots against, so for my model the “rates” version is obviously:

dSv% = Sv% – xSv% = GSAx/SA

Seasonal talent distribution

Now we can look at the actual talent distributions. Firstly, I will look at seasonal data and I will compare 4 different “rates” metrics: Classic Sv%, dSv% from Hockey-Statistics, dFSv% from Evolving-Hockey and dFSv% from TopDownHockey.

Here are those 4 metrics as a function of percentile. The minimum requirement is 700 fenwicks against (around 16-20 games) and that gives us 755 datapoints from 2007-2021:

The distributions look mostly similar, but the player rankings are not necessarily similar. Let’s also look at the histograms.

In general, we see distributions skewed towards the bad goaltenders. So, it seems like bad goaltenders have a greater impact than good goaltenders. How much the distribution is skewed depends on the model, but overall it looks as if you will lose more from bad goaltending than you will gain from good goaltending.

That being said, goaltenders probably shouldn’t be judged too much based on single season results, because there’s so much variance in goaltender data!

I also looked at talent/impact vs workload (graph below) and it showed two things:

  1. There’s a slight correlation between workload and positive impact. You would probably expect a greater difference between starters and backups though.
  2. The variance decreases with workload, so 700 fenwicks against is likely too small of a sample size.

The talent vs. workload graph is based on my model, but the graphs look the same for the other models.

Career talent distribution

Goaltenders are really tough to evaluate, because there’s so much variance from year to year. So, if we want to look at talent distribution, it’s probably better to look at career data.

Here’s the talent distributions for goaltenders with at least 4000 fenwicks against in their career (around 100 games). This gives us 103 goaltenders and Alexandar Georgiev just made the cut.

I’ve added a trendline to show how linearly distributed goaltenders are, when we increase the sample size. There’s still a big difference between a good goaltender and a bad goaltender, but I’m not sure there is an elite tier like we saw for the skaters. Especially, since the top goaltenders are different from model to model.

The top goalies

Here’s the top 15 goaltenders in terms of Sv% and how they perform according to the other models:

PlayerSv%HS_dSv%EH_dFSv%TDH_dFSv%
Tim Thomas92.39%1.01%0.22%0.66%
Tuukka Rask92.18%0.55%0.17%0.53%
Tomas Vokoun92.15%0.91%0.20%0.50%
Ben Bishop92.06%0.56%0.27%0.39%
Philipp Grubauer92.05%0.98%0.49%0.77%
Juuse Saros92.02%1.07%0.54%0.62%
Andrei Vasilevskiy91.98%0.60%-0.15%0.20%
Antti Raanta91.96%0.87%0.54%0.67%
Roberto Luongo91.84%0.47%0.19%0.39%
Robin Lehner91.84%0.29%-0.03%0.16%
Cory Schneider91.82%0.86%0.37%0.50%
Henrik Lundqvist91.82%0.48%0.87%0.29%
Corey Crawford91.79%0.64%0.30%0.17%
Anton Khudobin91.75%0.22%0.22%0.30%
Carey Price91.71%0.40%0.18%0.41%

And here’s how they rank in each model

Player Sv% HS_dSv% EH_dFSv% TDH_dFSv%
Tim Thomas12173
Tuukka Rask212256
Tomas Vokoun34207
Ben Bishop4111313
Philipp Grubauer5351
Juuse Saros6134
Andrei Vasilevskiy794728
Antti Raanta8542
Roberto Luongo9172214
Robin Lehner10243831
Cory Schneider11678
Henrik Lundqvist1215120
Corey Crawford1381230
Anton Khudobin14281819
Carey Price15222411

I’ve also looked at total GSAx and how it’s distributed. I used GSAA as the “totals” version of Sv%.

In the Evolving-Hockey graph, we see Henrik Lundqvist being so much better than anyone else that he completely bursts the scale. This is what happens when one of the best goalies in the league play under the best possible rink bias circumstances.

Other than that, we clearly see a distribution skewed towards the better players. This would indicate that above average goaltenders have long careers as starters while goaltenders below average have shorter careers or play in a more limited role.

To show this we can look at dSv% (talent) as a function of fenwicks against (indication of career length/role).

The top goalies

It’s worth looking at the top 15 GSAA goalies, and see how they perform in the other models. Just as a way to compare the models:

Player GSAAHS_GSAx EH_GSAx TDH_GSAx
Tuukka Rask138.285.536.7112.7
Henrik Lundqvist128.0106.8268.590.7
Tim Thomas114.795.927.482.8
Roberto Luongo109.685.247.695.5
Carey Price107.083.551.4118.4
Tomas Vokoun104.788.126.466.7
Pekka Rinne90.515.0-26.6-19.5
Ben Bishop88.463.842.062.3
Andrei Vasilevskiy78.054.7-18.724.9
Marc-Andre Fleury76.799.830.471.4
Ryan Miller69.0105.62.660.6
Corey Crawford68.489.156.331.5
Sergei Bobrovsky64.668.947.676.6
Jaroslav Halak62.579.988.787.9
Semyon Varlamov62.170.130.280.8

And here are the rankings:

Player GSAA HS_GSAx EH_GSAx TDH_GSAx
Tuukka Rask18182
Henrik Lundqvist2114
Tim Thomas35236
Roberto Luongo4993
Carey Price51071
Tomas Vokoun672413
Pekka Rinne7296771
Ben Bishop8141515
Andrei Vasilevskiy9165328
Marc-Andre Fleury1032111
Ryan Miller1123418
Corey Crawford126526
Sergei Bobrovsky1313109
Jaroslav Halak141125
Semyon Varlamov1512227

Predictability

Goaltending is by far the most frustrating part of hockey statistics, and it all comes down to predictability (or the lack thereof). The above analysis was about descriptive goaltender modelling. All of the metrics (including regular Sv%) tries to measure impact of the goaltenders.

Now we will look at how well these metrics predict future performances. To do this we simply compare impact in one year to the impact the following year. The minimum requirement is once again 700 FA in both seasons.

Here’s the results:

There’s very little correlation between performance one year and performance the next year. This means it’s nearly impossible to predict future goaltender performances based on the current available data.

We see that EH_dFSv% is more repeatable than the rink bias adjusted metrics from Hockey-Statistics and TopDownHockey. This isn’t really surprising since rink bias is relatively consistent from year to year. So, adjusting for rink bias may decrease the repeatability, but it should definitely increase the descriptive quality of the model.

The lack of predictability raises a few questions.

Does GSAx do a good job of measuring goaltender impact? Expected goals models are based primarily on shot location data, but they don’t factor in pre-shot puck movement. This makes you wonder if GSAx really is as good a goalie metric as some make it out to be. I don’t know the answer.

This leads me another question. Are goaltender performances really this random? You can see a goalie be the best one year and sub-replacement level the very next year. This seems weird. Theoretically, goalie performances could be much more consistent than current stats indicate. The stats just don’t catch it.

The reason you use dFSv% instead of Sv% is because you want to isolate goaltender performance from team defense, but it’s probably naive to think that modern goaltender stats can completely isolate goaltending from team defense.

Goaltender impact

The last thing I will do in this article is to try and compare goaltender impact to skater impact. To do this I will look at absolute values of GSAx and compare it sGAA.

Both my sGAA model and GSAx models are calibrated so that the average is always 0 and 1 GSAx/sGAA is worth one goal in team goal differential. This means that we should be able to compare the two models directly.

By looking at absolute values, we see how much the skaters/goalies differ from average – We see their impact regardless if it’s good or bad.

Here’s the seasonal absolute sGAA/GSAx values for all players

Position |GSAx/sGAA| |GSAx/sGAA|_60
Goalies79400.242
Forwards224090.232
Defenders116450.181

We see that forwards have the greatest overall impact, but when we look at impact per 60, goalies have the bigger impact. Considering goalies also play more, they have by far the greatest seasonal impact. To show this I’ve looked at the top 20 GSAx/sGAA seasons:

Player Season sGAA/GSAx
Tim Thomas10-1140.50
Tim Thomas08-0939.77
Mike Smith11-1236.83
Ryan Miller09-1036.28
Sergei Bobrovsky16-1735.00
Carey Price14-1531.35
Tomas Vokoun08-0930.81
Carey Price10-1127.71
Pavel Datsyuk07-0825.91
Cory Schneider11-1225.77
Roberto Luongo10-1125.73
Semyon Varlamov13-1425.49
Connor Hellebuyck19-2025.36
Ben Bishop13-1425.34
Corey Crawford15-1625.33
Tomas Vokoun09-1024.47
Cory Schneider14-1524.42
Jaroslav Halak18-1923.82
Brian Elliott15-1623.82
Marc-Andre Fleury20-2123.73

This list consists of 19 goalies and just 1 forward. However, we’ve already shown that there’s very little predictability when it comes to goaltending. So, what happens when we look career impact instead? Here’s the top 20 GSAx/sGAA careers:

PlayersGAA/GSAx
Sidney Crosby147.09
Pavel Datsyuk132.37
Henrik Lundqvist106.81
Ryan Miller105.57
Alex Ovechkin103.22
Brad Marchand101.60
Steven Stamkos101.59
Anze Kopitar101.05
Joe Thornton100.10
Marc-Andre Fleury99.76
Joe Pavelski99.54
Jonathan Toews99.41
Cory Schneider98.16
Evgeni Malkin96.33
Tim Thomas95.92
Mark Giordano92.45
Connor McDavid90.54
Corey Crawford89.07
Tomas Vokoun88.06
Claude Giroux86.86

Now, we see a combination of forwards and goalies (+Giordano). So, when we just look at single season impact, goaltenders have a greater impact than skaters. But when we increase the sample size and look at career numbers, then the best forwards have a similar (or greater) impact than the best goalies.

Summary

  • Goaltender performances are almost impossible to predict. It’s difficult so say, if this is due to bad models/data or if goaltending performances just are this random.
  • On a seasonal basis the talent distribution is skewed towards the bad goaltenders, indicating that the negative impact of the bad goalies is greater than the positive impact of the good goalies. When looking at career stats the differences are smaller, and almost linearly distributed.
  • When you look at total GSAx, there is a pretty good correlation between talent and career length/workload.
  • Goaltending is obviously extremely important, but it’s difficult to predict, so overcommiting to one goaltender is a bad idea, unless you’re very confident in his ability. It may be preferable to go with an 1A/1B tandem, since it gives you two chances for a great performance. Very few goalies are good year after year.
  • Goaltenders have greater seasonal impact (good or bad) than skaters, but not necessarily greater career impact.

Data from Evolving-Hockey and TopDownHockey

One thought on “Talent distribution – Goaltending (Part IV)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: