Blog 6: Player evaluation – The Goaltenders

Next up is the goaltenders – the Achilles Heel of every hockey statistician. There are generally two ways to judge goalies. You either go with goals saved above average (GSAA) or you go with goals saved above expected (GSAx) which is based on xGA. GSAA relies solely on the number of shots against and the number of goals allowed. GSAx is the difference between expected goals against and actual goals against, so it depends on the number of shots and the quality of the shots. The problem is that xG-models aren’t perfect when it comes to evaluating goaltending.

My model is basically GSAx with some slight adjustments. Here’s the top 20 goalies since season 2007/2008:

Player	GP	Sv%	GSAA	GSAx	LS-GAA_GK
Henrik Lundqvist	764	91.78	127.98	268.46	284.65
Jaroslav Halak	504	91.68	64.09	95.93	105.57
John Gibson	287	91.83	53.30	81.29	86.68
Carey Price	682	91.74	111.27	58.55	83.76
Jonathan Quick	644	91.34	18.49	53.08	67.82
Roberto Luongo	626	91.83	109.55	47.62	66.97
Sergei Bobrovsky	507	91.71	66.57	58.12	66.55
Braden Holtby	468	91.60	44.35	56.11	62.40
Jonas Hiller	404	91.44	21.59	52.61	61.87
Corey Crawford	486	91.78	68.38	56.29	60.34
Cory Schneider	409	91.80	54.87	58.17	58.13
Frederik Andersen	369	91.65	46.06	49.77	56.62
Cam Ward	611	91.03	-30.56	35.12	55.06
Cam Talbot	314	91.51	20.67	49.39	49.43
Tomas Vokoun	316	92.16	104.68	26.39	48.07
Ben Bishop	413	92.05	88.40	41.95	48.04
Evgeni Nabokov	344	91.17	5.24	26.14	46.60
Antti Raanta	186	92.06	41.60	43.57	46.11
Tim Thomas	317	92.36	114.66	27.35	44.15
Tuukka Rask	536	92.18	134.88	36.47	43.21

I already discussed Lundqvist back in blog 3. While he’s obviously a great goalie, he still seems a bit overrated when it comes to GSAx/LS-GAA. New York Rangers historically has a bad xG against, which theoretically makes it “easier” for Lundqvist to have a good GSAx. Most of this is obviously due to the team play of the Rangers, but part of it could also be due to rebound control. If the goalie gives up a lot of rebounds it naturally leads to a bigger xGA, and perhaps a better GSAx. So even though bad rebound control isn’t a good goalie trait, it could actually be a good thing for your GSAx. This could be part of the explanation regarding Lundqvist.

Here’s the best goalies in terms of GSAA. Lundqvist is still great, but now he’s human.

Player	GP	Sv%	GSAA	GSAx	LS-GAA_GK
Tuukka Rask	536	92.18	134.88	36.47	43.21
Henrik Lundqvist	764	91.78	127.98	268.46	284.65
Tim Thomas	317	92.36	114.66	27.35	44.15
Carey Price	682	91.74	111.27	58.55	83.76
Roberto Luongo	626	91.83	109.55	47.62	66.97
Tomas Vokoun	316	92.16	104.68	26.39	48.07
Pekka Rinne	657	91.73	91.28	-16.58	-0.23
Ben Bishop	413	92.05	88.40	41.95	48.04
Ryan Miller	650	91.62	80.15	13.42	37.35
Corey Crawford	486	91.78	68.38	56.29	60.34
Sergei Bobrovsky	507	91.71	66.57	58.12	66.55
Jaroslav Halak	504	91.68	64.09	95.93	105.57
Robin Lehner	301	91.85	57.16	-9.80	-2.21
Andrei Vasilevskiy	260	91.90	57.04	-32.39	-21.68
Marc-Andre Fleury	708	91.49	56.56	10.27	34.54
Cory Schneider	409	91.80	54.87	58.17	58.13
John Gibson	287	91.83	53.30	81.29	86.68
Frederik Andersen	369	91.65	46.06	49.77	56.62
Connor Hellebuyck	269	91.71	45.64	27.77	37.41
Braden Holtby	468	91.60	44.35	56.11	62.40

So what list ranks the goaltending best? The truth probably lies somewhere in between GSAA and GSAx. I honestly don’t think there’s any good way to measure goaltending as of right now. It’s almost impossible to distinguish between good goaltending and good team defense. The new upcoming tracking data may lead to a better goalie stat. Until then, this is what we got.

Here’s the players with the biggest positive difference between LS-GAA and GSAA. The worst teams in terms xGA per 60 minutes are: Edmonton, Islanders, Rangers and Carolina. Those are also the teams dominating this list.

Player	Season	Team	GP	Sv%	GSAA	LS-GAA	Diff
Henrik Lundqvist	09-10	NYR	73	92.08	20.17	51.52	31.35
Henrik Lundqvist	08-09	NYR	70	91.63	15.69	46.30	30.61
Henrik Lundqvist	07-08	NYR	72	91.19	5.28	28.23	22.95
Jaroslav Halak	14-15	NYI	59	91.39	-1.18	18.17	19.35
Ondrej Pavelec	13-14	WPG	57	90.09	-21.45	-4.18	17.27
Nikolai Khabibulin	10-11	EDM	47	88.98	-31.93	-14.87	17.06
Cam Ward	09-10	CAR	46	91.47	4.68	21.44	16.76
Kari Lehtonen	16-17	DAL	59	90.24	-17.17	-0.98	16.19
John Gibson	18-19	ANA	58	91.68	12.83	29.00	16.17
Kari Lehtonen	13-14	DAL	65	91.9	9.55	25.00	15.45
Devan Dubnyk	11-12	EDM	47	91.45	1.33	15.55	14.22
Cam Talbot	16-17	EDM	73	91.93	12.31	26.42	14.11
Ondrej Pavelec	11-12	WPG	68	90.62	-14.94	-0.97	13.97
Cam Ward	07-08	CAR	69	90.37	-9.90	3.91	13.81
Dwayne Roloson	09-10	NYI	50	90.65	-7.44	6.14	13.58
Evgeni Nabokov	13-14	NYI	40	90.51	-9.58	3.69	13.27
Henrik Lundqvist	11-12	NYR	62	92.98	28.59	41.09	12.50
Cam Ward	11-12	CAR	68	91.51	3.31	15.76	12.45
Ondrej Pavelec	14-15	WPG	50	92.02	7.50	19.67	12.17
Nikolai Khabibulin	11-12	EDM	40	91.02	-3.67	8.48	12.15

We can also look at the other end of the spectrum. Here’s the goalies with the biggest negative difference between LS-GAA and GSAA. The teams with the lowest xGA per 60 are: Minnesota, Boston, New Jersey, St. Louis and Nashville.

Player	Season	Team	GP	Sv%	GSAA	LS-GAA	Diff
Devan Dubnyk	16-17	MIN	65	92.35	18.35	-9.86	-28.21
Andrei Vasilevskiy	17-18	T.B	65	91.95	15.09	-12.00	-27.09
Niklas Backstrom	08-09	MIN	71	92.28	29.45	3.60	-25.85
Robin Lehner	16-17	BUF	59	92.05	13.41	-9.60	-23.01
Devan Dubnyk	17-18	MIN	60	91.8	10.23	-11.80	-22.03
Niklas Backstrom	07-08	MIN	57	91.81	14.54	-6.05	-20.59
Devan Dubnyk	18-19	MIN	67	91.32	6.17	-13.30	-19.47
Tim Thomas	11-12	BOS	59	92.04	11.46	-7.80	-19.26
Tuukka Rask	13-14	BOS	58	92.99	26.29	7.07	-19.22
Jimmy Howard	09-10	DET	63	92.37	23.10	4.18	-18.92
Niklas Backstrom	09-10	MIN	60	90.33	-12.96	-31.50	-18.54
Tuukka Rask	14-15	BOS	70	92.24	15.67	-1.45	-17.12
Andrei Vasilevskiy	16-17	T.B	50	91.69	5.03	-11.67	-16.70
Pekka Rinne	11-12	NSH	73	92.29	20.18	3.99	-16.19
Pekka Rinne	14-15	NSH	64	92.25	14.26	-1.93	-16.19
Sergei Bobrovsky	13-14	CBJ	58	92.29	15.28	-0.78	-16.06
Roberto Luongo	14-15	FLA	61	92.08	10.79	-4.56	-15.35
Steve Mason	11-12	CBJ	46	89.45	-25.83	-40.66	-14.83
Tim Thomas	10-11	BOS	57	93.82	45.85	31.17	-14.68
Ben Bishop	15-16	T.B	61	92.65	19.47	5.04	-14.43

My point is some goalies are likely overvalued if you judge them by LS-GAA or GSAx, while others are undervalued. I don’t think GSAA is any better though, so perhaps the smartest way to evaluate goalies is by combining the two. That’s something for another blog. Let’s instead turn our attention to this season.

Season 19/20:

Player	GP	Sv%	GSAA	LS-GAA
Connor Hellebuyck	58	92.20	22.35	26.18
Darcy Kuemper	29	92.80	16.63	11.79
Tuukka Rask	41	92.85	22.48	11.60
Corey Crawford	40	91.66	9.06	10.84
Mikko Koskinen	38	91.74	9.13	10.06
James Reimer	25	91.43	3.36	9.70
Jake Allen	24	92.67	11.21	9.39
Antti Raanta	33	92.11	11.66	9.19
Ben Bishop	44	91.99	13.25	8.93
Carter Hart	43	91.36	4.44	8.76
Anton Khudobin	30	92.97	17.73	8.70
Chris Driedger	12	93.79	10.00	8.40
Jordan Binnington	50	91.20	3.45	8.26
Robin Lehner	36	91.98	12.64	7.76
Jacob Markstrom	43	91.76	11.36	7.75
Igor Shesterkin	12	93.16	9.33	7.44
Calvin Petersen	8	92.19	3.32	7.28
Pavel Francouz	34	92.25	12.85	6.62
Henrik Lundqvist	30	90.50	-4.09	6.56
Alexandar Georgiev	34	90.97	0.09	6.55

Hellebuyck has been pretty great this season, especially when looking at LS-GAA. If you look at Sv% he has still been good, but not quite as good. It’s also worth noting that a true starter like Hellbuyck is rare these days. Most teams go with an 1A/1B situation and that strategy seems to be working for a lot of teams.

I have also listed best goalies measured by GSAA:

Player	GP	Sv%	GSAA	LS-GAA
Tuukka Rask	41	92.85	22.48	11.60
Connor Hellebuyck	58	92.20	22.35	26.18
Anton Khudobin	30	92.97	17.73	8.70
Darcy Kuemper	29	92.80	16.63	11.79
Ben Bishop	44	91.99	13.25	8.93
Pavel Francouz	34	92.25	12.85	6.62
Robin Lehner	36	91.98	12.64	7.76
Andrei Vasilevskiy	52	91.71	12.09	-1.53
Elvis Merzlikins	32	92.27	12.07	2.81
Antti Raanta	33	92.11	11.66	9.19
Jacob Markstrom	43	91.76	11.36	7.75
Jake Allen	24	92.67	11.21	9.39
Tristan Jarry	33	92.06	10.77	4.38
Chris Driedger	12	93.79	10.00	8.40
Igor Shesterkin	12	93.16	9.33	7.44
Mikko Koskinen	38	91.74	9.13	10.06
Corey Crawford	40	91.66	9.06	10.84
Jaroslav Halak	31	91.93	8.81	6.18
Cam Talbot	26	91.92	7.51	3.07
Mackenzie Blackwood	47	91.46	7.26	4.68

That’s it for now. Goaltending is very difficult to measure, and LS-GAA is in no way perfect. It does however fit with the other LS-GAA components, so that the overall result is right. In other words, LS-GAA_Total describes team results really good, but that doesn’t necessarily mean that the goaltending component tells the truth.

Next time I will divide players into tiers using LS-GAA per 60, and use these tiers to discuss how to best construct a team.

Conclusion:

• Goaltending can’t be effectively measured – that’s my belief.
• Goaltending is more or less unpredictable with the current metrics, but at the same time goaltending is hugely important. So if you can somehow predict future goaltender performance, then you can make a great projection model.

Stay safe and remember to be kind

All raw data from evolving-hockey.com