The Most Consistently Excellent NHL Goalie: Patrick Roy

One thing the data reveals about NHL goalies is that most NHL goalies are consistently inconsistent overall.  But not all are.  Now I want to turn to looking at the NHL goalies that are consistently consistent.  How do measure it?  I propose the following:  for an individual season, each NHL goalies performance using WAA is "graded" as follows: if they are in the top 20% of all NHL goalies for that season, they get an A, if they are in the next 20%, they get a B, and so on.  Then an NHL goalie is consistent if they maintain their grade from one season to another.  But I only am interested in the highest level of consistent performance, so I only count NHL goalies that from one season to another are among the top 20% of NHL goalies from the 1955/56 season to the 2019/20 season.  Thus, they have to be consecutively in the top 20% of NHL goalies for each season to be counted on the list.

The most consistently excellent NHL goalie performance over their career was by Patrick Roy.

Here are the Top 10 Consistently Excellent NHL Goalies

1. Patrick Roy (16):  1986/87-2002/03
2. Tony Esposito (11):  1969/70-1980/81
3. Billy Smith (9):  1974/74-1983/84
-. Dominic Hasek (9):  1992/92-2001/02
5. Glenn Resche (7):  1974/75-1981/1982
-. Henrik Lundquist (7):  2008/09-2015/16
-. Roberto Luongo (7):  2000/01-2008/09
8. Felix Potvin (5):  1991/92-1996/97
-. Jean-Sebastien Giguere (5): 2001/02-2007/08
-. Tomas Vokum (5):  2005/06-2010/11

The Greatest NHL Goalie: Ken Dryden

Recently, I posted that Bernie Parent was the greatest NHL goalie (since 1955/56, the season where the NHL started to record shots against, which I need to make the calculation of goalie production).  Well leave it to an economist to provide more than one answer to one question - Who is the greatest NHL goalie?

Bernie Parent had the highest Wins Above Average (WAA) for a single season, and that is one way of evaluating who the greatest (or most productive) NHL goalie ever was. 

Yet, there are other ways, and one of those is to take a look at career WAA and find out who had the highest career average WAA.  Running the number since the 1955/56 season, the NHL goalie with the highest average WAA was Ken Dryden (by a large margin).

Here is a list of the Top 10 NHL goalies in terms of career WAA:

1.   Ken Dryden
2.   Tony Esposito
3.   Dominik Hasek
4.   Patrick Roy
5.   Bernie Parent
6.   Jacques Plante
7.   Johnny Bower
8.   Tim Thomas
9.   Roberto Luongo
10. Glenn Hall

The Greatest NHL Goalie: Bernie Parent

What is the "greatest"?  Greatest could be the absolute best in terms of some metric.  Greatest could be the most times in the top 10/20/25/100 using some metric.  Greatest could be the most consistent top performances.  Greatest could be ...

Today I am going to attempt to answer this question:  Who is the Greatest NHL Goalie in terms of productivity?  So how is goalie productivity measured?  In a paper published in the Journal of Sports Economics, David Berri and I came up with a way of measuring NHL Goalie production using a concept called Wins Above Average (WAA).  WAA uses save percentage as the main determinant of goalie productivity.

Save percentage is a simple and straightforward way of measuring NHL goalie productivity.  Other measures such as, wins (often used) is more of a team measure, while goals against average is also more of a defensive measure.  When all else fails, how well does the goal keep a shot on goal out of the net; that is our measure of goalie productivity.  You are more than welcome to quibble with this idea, but it is simple and easily justified; so I am staying with it.

WAA is measured as:  the absolute value (since a goal against has a negative effect on team wins) of the marginal value of a goal against divided by two (since each win is worth two standings points) times, the number of shots on goal that goalie faces times the difference in the save percentage of the goalie and the average save percentage of all goalies for that season.

Given the NHL started to collect Shot Attempts beginning with the 1955/56 season, I have collected all the NHL goalie data (from NHL.com) going back to the 1955/56 season to the present.  Then I calculated each season's average save percentage; and then calculated WAA for the nearly 4000 goalies that played in the NHL since the 1955/56 season.  Ranking the NHL goalies from highest to lowest results in Bernie Parent to have the highest WAA since 1955/56.  Using a GA coefficient of -0.34, yields Bernie Parent's WAA = 12.282 for the Philadelphia Flyers in the 1973/74 season (the "Flyers Win the Stanley Cup" as announced by Gene Hart).

2013-2014 NHL Regular Season Goalie Consistency

At the end of the regular season I ran the numbers for NHL goalies and found that Tuukka Rask, Semyon Varlamov and Carey Price were the top three NHL goalies for this season using Wins Above Average (WAA).  WAA is the marginal value of a goal against divided by two (since each win is worth two standings points) times the number of shots on goal that a goalie faces times the difference in the save percentage of the goalie and the average save percentage of all goalies for that season in absolute value (since a goal against has a negative effect on team wins).  Here is a step-by-step guide to calculate NHL goalie performance.

Now what I want to do is figure out how consistent NHL goalies are from one season to another, so I am comparing the regular season performance from last season (which was shortened due to the lockout) and this season.  To do that, I am going to only look at goalies that started at least 10% of the regular season games played for each season, since the number of games played is different between the two regular seasons.

Thus to find how consistent NHL goalies are in consecutive regular seasons,  I am looking at the percentage of NHL goalies who are in the same WAA quantile in consecutive regular seasons and those who have moved up or down one quantile from one season to the next.  Here is a step-by-step guide on measuring player consistency.

What I find is that of the NHL goalies who started at least 10% of the total number of regular season games, 30.8% were consistent (in the same quantile in each season), 32.7% were near consistent (either moved up or down one quantile from one regular season to another) and 36.5% were inconsistent (moved two or more quantiles from one regular season to another).

Of those that were consistent, there were only three NHL goalies who were in the top quantile (i.e. top 20% of NHL goalies in consecutive regular seasons).  They are Tuukka Rask (Boston Bruins), Sergei Bobrovsky (Columbus Blue Jackets) and Henrik Lundqvist (New York Rangers).  Given that Tuukka Rask was the most productive goalie (but just barely) and that he is also only one of three goalies to be in the top 20% from one season to the next, I would say that he is the best candidate (statistically) for the Vezina Trophy.

NHL Regular Season Goalie Performance for 2013-14

For the past few years I have been looking at NHL goalies and evaluating their regular season performance. So now that the 2013/14 NHL regular season is in the books, let's look at NHL goalie performance using our NHL goalie measure called Wins Above Average.  For those interested in how I calculate this measure, here is a step-by-step guide to measure WAA.  As mentioned in the previous link, I will need a measure of the impact that goals against has on team points, and for the 1995 to 2013 NHL regular seasons, this is equal to -0.340858.

So who is the best NHL goalie for the regular season?  Tuukka Rask, followed very closely by Semyon Varlamov.  Here are the top 20 goalies this season ranked by WAA.

	Player	Team		WAA		SV%
1	Tuukka Rask	BOS		4.521		0.930
2	Semyon Varlamov	COL		4.517		0.927
3	Carey Price	MTL		4.102		0.927
4	Ben Bishop	TBL		3.046		0.924
5	Jonathan Bernier	TOR		2.792		0.923
6	Sergei Bobrovsky	CBJ		2.654		0.923
7	Cam Talbot	NYR		2.593		0.941
8	Josh Harding	MIN		2.254		0.933
9	Anton Khudobin	CAR		2.231		0.926
10	Henrik Lundqvist	NYR		1.902		0.920
11	Alex Stalock	SJS		1.768		0.932
12	Martin Jones	LAK		1.718		0.934
13	Jaroslav Halak	STL, WSH		1.692		0.921
14	Ben Scrivens	LAK, EDM		1.677		0.922
15	Kari Lehtonen	DAL		1.662		0.919
16	Roberto Luongo	VAN, FLA		1.399		0.919
17	Cory Schneider	NJD		1.352		0.921
18	Chad Johnson	BOS		1.347		0.925
19	Ryan Miller	BUF, STL		1.338		0.918
20	Frederik Andersen	ANA		1.223		0.923

NHL Goalie Performance at the Olympic Break

Well, I am back.  To mark my re-emergence I have decided to look at how NHL goalies have performed up to the Olympic break.  Below are the top 20 NHL goalies using our Wins Above Average measure.

Rank	Player	Team	WAA	GS	SA	SV%
1	Ben Bishop	TBL	3.968	44	1229	0.933
2	Jonathan Bernier	TOR	3.187	38	1444	0.927
3	Tuukka Rask	BOS	2.877	43	1210	0.928
4	Carey Price	MTL	2.724	48	1459	0.925
5	Ben Scrivens	LAK, EDM	2.717	21	695	0.937
6	Semyon Varlamov	COL	2.436	43	1436	0.924
7	Josh Harding	MIN	2.228	26	690	0.933
8	Ryan Miller	BUF	2.096	39	1373	0.923
9	Cory Schneider	NJD	1.630	31	800	0.926
10	Cam Talbot	NYR	1.602	15	449	0.935
11	Martin Jones	LAK	1.547	12	350	0.94
12	Alex Stalock	SJS	1.440	11	353	0.938
13	Anton Khudobin	CAR	1.245	19	564	0.927
14	Frederik Andersen	ANA	1.151	16	484	0.928
15	Marc-Andre Fleury	PIT	1.081	47	1278	0.919
16	Thomas Greiss	PHX	1.077	10	352	0.932
17	Philipp Grubauer	WSH	0.933	14	458	0.926
18	Al Montoya	WPG	0.924	15	495	0.925
19	Steve Mason	PHI	0.856	43	1267	0.918
20	Henrik Lundqvist	NYR	0.839	43	1243	0.918

Martin Biron Retires

The USA Today reports that Martin Biron has retired from the NHL.  So I thought that I would take a look back at his NHL career using our NHL goalie performance measure called Wins Above Average (WAA).  Here is Martin Biron's goals against average (GAA) and his WAA over his career.

 

Notice that in terms of GAA, his numbers are fairly similar, but in terms of WAA his numbers move around much more.  The Pearson correlation between GAA and WAA is -0.323, indicating that GAA and WAA weakly move in opposite directions.  One of the concepts that we are interested in is how do NHL goalies perform from year-to-year, or how consistent are NHL goalies. Earlier this month I reported that NHL goalies remain "consistent" only about 25% of the time covering the 1997-98 to 2012-13 NHL regular seasons.  To help with that, here is where Martin Biron ranked each season in terms of Wins Above Average (remember that lower numbers are higher or better ranking).

As you can tell from the graph above, Biron was rather inconsistent in terms of his overall WAA, especially over the end of his career.

NHL Goalie Consistency Revisited

With the 2013-2014 NHL regular season underway today, I thought it would be a good time to look at how consistent NHL goalies have been since the 1997 regular season including data through last regular season.  In order to do this, I looked at NHL goalies who started at least 10% of the regular season games for a given season for two consecutive seasons.  That is to take care of issues where goalies are injured or were not considered by the team to be productive enough to be an integral part of the team.  While the 10% cut-off may seem to high or too low, it is a good starting point to use so that I do include both the starting and back-up goalies for a given team in the analysis.

Thus, what I need is a measure for evaluating NHL goalie performance and a way of measuring consistency from one season to another.

In a paper published in the Journal of Sports Economics, David Berri and I measured NHL goalie performance. WAA is our measure of NHL goalie productivity and is based on the absolute value (since a goal against has a negative effect on team wins) of the marginal value of a goal against divided by two (since each win is worth two standings points) times the number of shots on goal that goalie faces times the difference in the save percentage of the goalie and the average save percentage of all goalies for that season.

To measure consistency, I ranked each goalie who had started at least 10% of the regular season games for that season and "graded" each goalie based on where they were in the ranking.  Specifically, all goalies that were in the top 20% were "graded" as an A, between 21% and 40% were "graded" as a B, and so on.  This is the same way we measured NFL QB's, MLB batters and NBA players in chapter 9 of The Wages of Wins.

Then I found out if that goalie started at least 10% of regular season games in consecutive regular seasons and evaluated whether they kept their same grade from one regular season to the next (which I called consistent), whether they moved up or down only one letter grade (which I call near consistent) or if they moved up or down more than two grades (which I call inconsistent).

From the 1997-98 regular season to the 2012-13 regular season, this happened 758 times (consecutive regular seasons starting at least 10% of the games for a goalie).  Of those 758 observations, NHL goalies kept the same grade about 25% of the time, moved up or down one grade about 35% of the time and moved two or more grade from one regular season to the next about 40% of the time.  Since all that I am looking at here is whether they kept the same grade (i.e. A's or F's in consecutive regular seasons), this seems to be a rather volatile amount of performance, and hence I would conclude that NHL goalies on the whole are rather inconsistent.

Of the 192 NHL goalies that kept their same grade, 34% were "A's", 13% were "B's", 18% were "C's", 14% were "D's" and 21% were "F's".  Of those that moved up or down one grade 16% of the total (121 observations) moved down one grade and 19% (or 141) moved up one grade.

Over the last two regular seasons performing the same analysis results in 52 observations and NHL goalies keeping their same "grade" 21% of the time, moving up or down one "grade" 33% of the time and moving two or more grades 46% of the time.  Again, lots of volatility from regular season to regular season.

NHL Goalie Ranking

ESPN has ranked current NHL goalies and so have I in the past.  So I thought that I would link to my latest NHL goalie ranking at the end of the 2013 regular season and re-post the top 20 here.

	Player	Team		WAA		SV%
1	Sergei Bobrovsky	CBJ		3.720		0.932
2	Craig Anderson	OTT		3.364		0.941
3	Henrik Lundqvist	NYR		2.863		0.926
4	Tuukka Rask	BOS		2.860		0.929
5	Antti Niemi	SJS		2.518		0.924
6	Cory Schneider	VAN		2.152		0.927
7	Jimmy Howard	DET		2.137		0.923
8	James Reimer	TOR		2.054		0.924
9	Corey Crawford	CHI		1.850		0.926
10	Robin Lehner	OTT		1.745		0.936
11	Devan Dubnyk	EDM		1.563		0.920
12	Braden Holtby	WSH		1.550		0.920
13	Viktor Fasth	ANA		1.025		0.921
14	Ben Bishop	OTT, TBL		0.968		0.920
15	Ray Emery	CHI		0.792		0.922
16	Jason LaBarbera	PHX		0.791		0.923
17	Chad Johnson	PHX		0.777		0.954
18	Kari Lehtonen	DAL		0.732		0.916
19	Nikolai Khabibulin	EDM		0.712		0.923
20	Ryan Miller	BUF		0.668		0.915

On opening day I will come back to NHL goalies and look at how consistent goalies have been over the last few years.

2013 NHL Goalie Performance

This past weekend I heard a number of hockey announcers state that the most important player in the playoffs was the goaltender.  That reminded me that I had not finished the regular season NHL goalie analysis.  For our purposes, the NHL goalie measure analyzes the number of wins above average a NHL goalie produces in a given time period (in this case the lockout shortened season). So here are the top 10 NHL goalies using our Wins Above Average measure of an NHL goalie.  The updated coefficient on goals against is -0.341814 over the seasons from 1995/96 to 2013.  Using that measure of the impact of goals against on standings points, yields the following top 20 NHL goalies during the 2013 regular season.

	Player	Team		WAA		SV%
1	Sergei Bobrovsky	CBJ		3.720		0.932
2	Craig Anderson	OTT		3.364		0.941
3	Henrik Lundqvist	NYR		2.863		0.926
4	Tuukka Rask	BOS		2.860		0.929
5	Antti Niemi	SJS		2.518		0.924
6	Cory Schneider	VAN		2.152		0.927
7	Jimmy Howard	DET		2.137		0.923
8	James Reimer	TOR		2.054		0.924
9	Corey Crawford	CHI		1.850		0.926
10	Robin Lehner	OTT		1.745		0.936
11	Devan Dubnyk	EDM		1.563		0.920
12	Braden Holtby	WSH		1.550		0.920
13	Viktor Fasth	ANA		1.025		0.921
14	Ben Bishop	OTT, TBL		0.968		0.920
15	Ray Emery	CHI		0.792		0.922
16	Jason LaBarbera	PHX		0.791		0.923
17	Chad Johnson	PHX		0.777		0.954
18	Kari Lehtonen	DAL		0.732		0.916
19	Nikolai Khabibulin	EDM		0.712		0.923
20	Ryan Miller	BUF		0.668		0.915

NHL Post-Season Goalie Evaluation for 2011-2012

Now that the NHL playoffs are over (congrats to the Los Angeles Kings), let's take a look at how the goalies performed during last season's playoffs.  I calculate goalies post-season performance in the same way as I have previously done, with the exception that I am calculate the coefficient on post-season goals against, instead of using the regular season in the Wins Above Average calculation.  Here are the results for the 2012 NHL playoffs.

Rank	Player	Team		WAA		SV%
1	Mike Smith	PHX		0.427		0.944
2	Jonathan Quick	LAK		0.416		0.946
3	Braden Holtby	WSH		0.196		0.935
4	Henrik Lundqvist	NYR		0.167		0.931
5	Cory Schneider	VAN		0.122		0.960
6	Craig Anderson	OTT		0.076		0.933
7	Pekka Rinne	NSH		0.071		0.929
8	Jaroslav Halak	STL		0.020		0.935
9	Tim Thomas	BOS		0.011		0.923
10	Johan Hedberg	NJD		0.003		0.929
11	Jake Allen	STL		0.000		0.000
12	Scott Clemmensen	FLA		-0.004		0.920
13	Jose Theodore	FLA		-0.010		0.919
14	Antti Niemi	SJS		-0.035		0.914
15	Brent Johnson	PIT		-0.048		0.667
16	Roberto Luongo	VAN		-0.061		0.891
17	Martin Brodeur	NJD		-0.086		0.917
18	Brian Elliott	STL		-0.102		0.904
19	Sergei Bobrovsky	PHI		-0.112		0.722
20	Jimmy Howard	DET		-0.121		0.888
21	Corey Crawford	CHI		-0.141		0.893
22	Ilya Bryzgalov	PHI		-0.351		0.887
23	Marc-Andre Fleury	PIT		-0.430		0.834

NHL Goalie Performance Measure

Here is a step-by-step guide to David Berri's and my NHL goalie measure from our article in the Journal of Sports Economics in 2010.

Step 1: Calculate the Marginal Value of a Goal Against. I have previously posted on how to do this, so follow the steps in this blog.
Step 2: Download NHL Goalie Data I downloaded the 2011-2012 NHL goalie data from NHL.com. Go to the web page, click on Stats, then choose Individuals. Once that page opens, I choose 2011-2012 Regular Season and under Position I choose Goalie. This opens up a page with all the NHL goalies for the time period you are analyzing. Make sure you get all the goalie data by choosing the additional tables at the bottom of the first table. I then copied and pasted each table into Microsoft Excel.
Step 3: Wins Above Average (WAA) Calculation. WAA is measured as the absolute value (since a goal against has a negative effect on team wins) of the marginal value of a goal against divided by two (since each win is worth two standings points) times the number of shots on goal that goalie faces times the difference in the save percentage of the goalie and the average save percentage of all goalies for that season.  Let's break this calculation up into three parts.
Part A:  This is already done, since the data is just the columns listed as SA and SV% from NHL.com for each goalie, which in my spreadsheet is column I for SA and column M for SV%. 
Part B:  Calculate the average save percentage for the league by finding the total number of shots against and the total number of goals against and then subtract the total number of goals against from the total number of shots against and divide that by the total number of shots against.  In Excel this would be (S6-S9)/S6, if total shots against is in cell S6 and total goals against is in cell S9.  Suppose this formula is put in cell S12.
Part C:  The first part of the equation [the marginal value of a goal against] was found in step 1.  Take the absolute value of this number and divide by two.  If the marginal value of a goal against is in cell S3, then in Excel type ABS($S$3)/2.
Put it all together and in Excel we have the following formula:  =(M2-$S$12)*I2*(ABS($S$3)/2).  Note that M2 is the first goalie listed save percentage and cell I2 is the first goalie listed shots against.  Then copy and paste this formula down in the spreadsheet for each goalie and you have an estimate of each goalie's Wins Above Average (WAA) for that season.
Step 4:  Rank Most Productive Goalies.  I then rank each goalie by their WAA from highest to lowest for that season.  You can do this by sorting from highest to lowest.  I do something slightly different but it is the same thing, it just allows me to do this without actually having to use the sort option in Excel.

NHL Regular Season Goalie Evaluation

Tonight starts the NHL playoffs, so I thought that I would take a look at how productive NHL goalies were during the 2011-2012 NHL regular season. In order to do so David Berri and I published a paper in the Journal of Sports Economics in 2010 on NHL goalies. We named our measure of NHL goalie productivity WAA. WAA the absolute value (since a goal against has a negative effect on team wins) of the marginal value of a goal against divided by two (since each win is worth two standings points) times the number of shots on goal that goalie faces times the difference in the save percentage of the goalie and the average save percentage of all goalies for that season. I am using -0.221519 for the marginal value of a goal. I plan on updating this value using data from the 2011-2012 season, but have not had the time. Note, any change in the marginal value will just be a monotonic transformation of the WAA (either increase or decrease) from its current value.

Given our measure of NHL goalie performance, here is the top 20 ranking for NHL goalies over the 2011-2012 regular season. (I hope to get some time to give a step-by-step guide as to how I am getting the WAA numbers later this month.)

Rank	Player	Team		WAA		SV%
1	Mike Smith	PHX		3.767		0.930
2	Henrik Lundqvist	NYR		3.196		0.930
3	Jonathan Quick	LAK		3.191		0.929
4	Brian Elliott	STL		2.849		0.940
5	Cory Schneider	VAN		2.456		0.937
6	Pekka Rinne	NSH		2.256		0.923
7	Miikka Kiprusoff	CGY		1.686		0.921
8	Jaroslav Halak	STL		1.672		0.926
9	Kari Lehtonen	DAL		1.630		0.922
10	Tim Thomas	BOS		1.187		0.920
11	Jimmy Howard	DET		1.071		0.920
12	Tuukka Rask	BOS		1.064		0.929
13	Roberto Luongo	VAN		0.954		0.919
14	Niklas Backstrom	MIN		0.786		0.919
15	Jose Theodore	FLA		0.576		0.917
16	Carey Price	MTL		0.522		0.916
17	Jean-Sebastien Giguere	COL		0.514		0.919
18	Tomas Vokoun	WSH		0.498		0.917
19	Ryan Miller	BUF		0.488		0.916
20	Justin Peters	CAR		0.451		0.931

NHL Goalie Evaluation at the 2012 All-Star Break

Changing gears (just for a today), let's take a look at NHL goalie evaluation at the All-Star Break.

As a reminder of what we are doing, David Berri and I published a paper in the Journal of Sports Economics in 2010 on how GM's evaluate NHL goalies. Basically what we found was that GM's are good at evaluating current player performance by using Vezina voting for the best NHL goalie and using past player performance in setting NHL salaries. Unfortunately, GM's on the whole are not so good at setting salaries in relation to the goalies future performance.

We named our measure of NHL goalie productivity WAA. WAA the absolute value (since a goal against has a negative effect on team wins) of the marginal value of a goal against divided by two (since each win is worth two standings points) times the number of shots on goal that goalie faces times the difference in the save percentage of the goalie and the average save percentage of all goalies for that season (or in this case up to the All-Star Break).

Running the numbers results in the following top 10 NHL goalies up to the All-Star Break.

Rank	Player	Team	WAA	SV%
1	Henrik Lundqvist	NYR	2.876	0.937
2	Jonathan Quick	LAK	2.839	0.934
3	Tim Thomas	BOS	2.082	0.933
4	Pekka Rinne	NSH	1.777	0.925
5	Brian Elliott	STL	1.676	0.938
6	Tuukka Rask	BOS	1.423	0.938
7	Jimmy Howard	DET	1.327	0.924
8	Mike Smith	PHX	1.135	0.922
9	Miikka Kiprusoff	CGY	0.914	0.92
10	Cory Schneider	VAN	0.822	0.927