In our book, The Wages of Wins, we look at how consistent players are from one season to another. Specifically we look at NBA players, MLB hitters and NFL quarterbacks. Unfortunately we skipped NHL players, a point I want to rectify.
So in order to figure out how consistent NHL goalies are from one season to another, I have collected NHL goalie regular season performance data for thirteen seasons (1997/98 to 2010/11) with the lockout season missing for obvious reasons. Then for each season, I have evaluated each NHL goalies regular season performance exactly as I did for the 2010/11 season earlier this year. For each season, I also eliminated from evaluation any goalie that started less than 10% of regular season games (i.e. less than 8 game starts), since these goalies are most likely emergency call ups or players that were injured for a significant part of the season. Including players with limited playing starts (and playing time) are likely to result in even greater performance variance, so including these goalies will make the consistency results poorer. So for that reason, I have limited the number of NHL goalies that had eight or more game starts in consecutive seasons.
Next, I "graded" all goalies in an individual season as those that are in the top 20% of WAA receive an A, those in the next highest 20% WAA receive a B all the way down to the bottom 20% of WAA who get an F. (Not that is a grading scale I would use in class). Then I looked at how many goalies over those thirteen seasons kept the same "grade" from one season to another. Those that keep the same "grade" (i.e. are in the same quantile) are considered consistent; those that move up or down only one quantile are considered nearly consistent and those that move up or down more than two quantiles are considered inconsistent. Some may argue that this is either too restrictive a measure for measuring consistency and others may feel this is too lenient.
For example, from 1997/98 to 2010/11 if a goalie started atleast 8 games each season they would have 12 observations of consecutive seasons. Martin Broduer has played during this entire time period. During this time period, Broduer has keep his same grade (top 20% each time) four times, and moved up or down one grade five times, meaning that is "grade" has changed by more than one "grade" three times.
Over the entire time period, for goalies that started at least eight games in consecutive seasons (including 2003/04 and 2005/06 as a consecutive season due to the lockout), NHL goalies as a whole maintained their "grade" - stayed in the same quantile - 26% of the time, and moved either up or down one "grade" 34% of the time; thus about 40% of the time NHL goalies moved up or down more than one "grade".
Thus, NHL goalies are about as consistent as NFL QB's, and are less consistent than MLB batters and much less consistent than NBA players.