Pro Football's All-Time Best Passers

Now that I've got a method for evaluating a quarterback as a passer and can use it compare his career to that of others, I thought it might be illustrative to compare the Top 50 on my list to how they rank on the all-time passer rating list provided by the Pro Football Hall of Fame.  The list provided by the Hall only includes those passers who've attempted 1,500 or more passes in their career.  Also, as of today (April 27, 2010), the Hall of Fame list does not include stats from the 2009 season.  As I've mentioned in the past, the HOF list unfairly penalizes those players who played in an era where passing was secondary.  Why?  Because of the restriction of limiting it to only those passers attempting 1,500 or more passes.  So, to supplement the list from the HOF, I'll show you their passer rating ranking based on my list of 416 passers, which is a broader and more inclusive list, especially of those who played in the 1930's - 1970's.  My list includes any quarterback who qualified in at least one season.

Here's the top 50 list of All-Time Best Passers:

My list of 50 not only covers the entire spectrum of pro football history - from the beginning in the 1930's to current players, but it also presents the players who played in earlier years fairly, and in context.  The question isn't whether Joe Montana was a better passer than Sammy Baugh, but, whether Joe Montana relative to Montana's peers, was a better passer than Sammy Baugh was relative to Baugh's peers.  The HOF list can be used to answer that question, since a measure relative to a fixed standard (in other words the passer rating system) can be used to do that.  However, the HOF list cannot be used to answer the next question - how does one compare Sammy Baugh to Joe Montana given that they both played an entirely different game, with different rules and circumstances.  A fixed standard cannot solve that particular puzzle.  The use of a mean and standard deviation by year (z scores) which in turn we convert to percentile ranks can be used.  C7 is simply a quarterback's 7 best years as a passer in terms of percentile rank summed up.

C Scores - A New Way to Evaluate Pro Football's Best Passers of All-Time

It has taken me two years, many, many iterations, and countless hours of mental anguish, but I think I've finally settled on a reliable method by which I can evaluate any quarterback's passing ability versus any other quarterback's passing ability regardless of when they played, or for how long they played.  So, for example, I can compare Peyton Manning versus Sammy Baugh, or, Otto Graham versus Brett Favre, or, Sonny Jurgensen versus Dan Marino.  You get the point.  It all started coming together for me as I wrote my post on Manning versus Brees versus Favre.  While I had already figured out how to effectively compare one quarterback's given season to that of another's given season, I had as yet to figure out an effective method by which we evaluated careers.

The inspiration came to me as I read Michael J. Schell's book, Baseball's All-Time Best Hitters, where he went about solving a different problem altogether, but had essentially the same characteristics of the problem I was trying to solve.  He was trying to account for the fact that for many hitters with long careers, their batting averages declined after they reached their mid-thirties.  And, he didn't think it was fair to compare a hitter who had played into their late thirties or early forties to a player that had not.  He chose to account for this by using a "late career adjustment".  What he did was to look only at a player's first 8,000 at-bats.  


I adopted this notion, and, instead of using a player's first x years, came up with the idea of using his best x years, given that a player's passing ability, as measured by CMTI, and compared to that of his peers tends to be a little volatile from year to year.


Deciding what x was going to be was an issue.  I wanted to balance outstanding achievement with duration.  In other words, I wanted to see who had performed at an exceedingly high level for a long time.  Remember, from my previous post, the average number of years a quarterback qualifies is 4.125.  The median is 3.  


I thought that a good starting point was to use 4 years.  This measure would include 169 players, or a little more than 40% of my database of 416 players going back to when the NFL officially started keeping statistics, and including both the AAFC in the 1940's and the AFL in the 1960's.


If I used 7 years, then I'd get 95 players, or almost 25% of the total.  That seemed reasonable for a couple of reasons.  First, 25% seemed like a good cut-off, albeit arbitrary.  Second, it would allow me to discard the worst years for those quarterbacks who had 8 or more years, which seemed like an appropriate way to reward longevity.  


I could have looked at 10 years - in which case, we would be limiting ourselves to 50 quarterbacks, or about 12% of the database, a rather small group, and, as we shall see, excluding many superb passers who didn't quite make it to 10 years.  Keep in mind that the median number of years a player qualifies, is 3 years.


My basic method for evaluating a player's performance in a given year is to look at his CMTI, and then, based on the mean and standard deviation for that given year, relate his performance to that of his peers.  Since the aggregate accumulation of these measures over the past 78 years looks very much like a normal (bell-shaped) curve, we can use the bell-curve to convert these measures to percentile ranks, giving a numerical value ranging from 1 to 99 for each of these performances.  The sum of a given quarterback's 4 best such performances are then used as my C4 calculation, and the sum of his 7 best performances are my C7 calculation.  Obviously, then the maximum possible value for C4 is 99 * 4 = 396, and, commensurately, the maximum C7 value is 99 * 7 = 693.  As I mentioned earlier, I've also calculated a C10 value for those players with an exceptionally long career.  Obviously, if we did this for every year a player qualified, one could create a career C score for each player.


Let's take a look at the lists.  We will start with C4.

Wow!  What a list.  Not bad for starters.  Remember, this is a ranking based on a quarterback's 4 best years as a passer.  Look at the top 14 on this list.  12 of them are in the Pro Football Hall of Fame!  Peyton Manning surely will be when he is done.  That's 13 out of 14!  Ken Anderson ranks 3rd all-time as a passer according to this list.  It is important to keep in mind that this is a list of the top-ranked passers, not top-ranked quarterbacks.  It is hard enought to determine a ranking of passers, so it would stand to reason then that it is much harder to determine what makes a top-ranked quarterback.  I don't know how the NFL goes about determining who gets in to the Hall of Fame and who deos not, but it sure seems odd that Ken Anderson is the only player in the top 14 on this list that isn't (assuming that Peyton is in, of course). 

One of the most aesthetically pleasing aspects of this list, especially looking at the top 15, is that every era and every decade is represented.  There does not seem to be a bias.  I would think that is an admirable quality of any top-ranked list in that it shows that the measure used to determine who is better, isn't necessarily biased towards a particular era - as opposed to say, the list of top ranked passers (see here and here) in terms of the NFL Passer Rating system at the Pro-Football Hall of Fame.  


Look at Steve Young's C4.  That's almost unbelievable!  The highest C4 possible is 396.  In other words, that is being in the 99th percentile four years.  Well, Young comes close.  His 1996 and 1997 seasons were in the 99th percentile, and his 1992 and 1994 seasons were in the 98th percentile.


In my previous post, we discussed Kurt Warner's merits as a passer.  We ignored any discussion about whether he will be enshrined in Canton.  Based on this list, it would appear that he is right on the fence.


There are a total of 27 quarterbacks on this list that are members of the Hall of Fame.  There are a total of 31 quarterbacks that have been inducted into the Hall of Fame.  The four that are in the Hall of Fame, but are not on this list are omitted for good reason.  They all played in the 'early years' of the NFL - prior to the keeping of official statistics.  They are, Jimmy Conzelman, who played from 1921-1929, John (Paddy) Driscoll, who played from 1920-1929, Earl (Dutch) Clark, who played from 1931-1938 and actually had two very good years in 1934 and 1936, and Benny Friedman, who played from 1927-1934, and who, many during his time argued was the finest passer of his time.  It is most unfortunate that we don't have official statistics that would show that.


OK, what about C7?

Woah!  An even better list than C4!  First, while the ordering changed, the top 15 stayed exactly the same as the C4 list, which probably suggests more of the power of the C4 list than anything else (in other words, a quarterback's best 4 years is a pretty good indicator of their passing ability).  However, now, we not only have 13 of the top 15 in the HOF (I am considering it a foregone conclusion that Peyton Manning will be inducted into the HOF), but 16 of the top 20 are either in the Hall of Fame or will be (I am also considering it a foregone conclusion that Brett Favre will also be inducted into the HOF).  That is a phenomenal list.  


It seems obvious then, that, in order to be inducted into the Hall of Fame as a quarterback, one must also have been a great passer.  Having said that, it's even more shocking to me now the omission of Ken Anderson from the HOF.  I simply do not understand why he is not in.  I will have to post an article on Ken Anderson in the near future.


What does C7 say about Kurt Warner?  Again, right on the fence.  

From my vantage point, it certainly appears that we should be discussing putting Ken Anderson in the Hall of Fame ahead of Kurt Warner.


Finally, we'll take a look at C10.  This list only includes 50 quarterbacks, and many exceptional passers are not on this list, simply because they didn't qualify in at least 10 years.

The story is similar to what we've already seen with the C4 and C7 lists.  14 of the top 15 (if you include Peyton Manning and Brett Favre as in the Hall of Fame) are or will be in the Hall of Fame.  The lone exception is - you guessed it - Ken Anderson.


There's several conclusions that I can draw from all of this.


I feel pretty good that I've come up with a method by which I can determine the all-time best passers in the history of professional football.  I also feel that this method will stand the test of time.  It will work regardless of how the game changes over time.  It is a method that allows me to conveniently compare players who played in different eras, and it allows me to compare players who have had different career lengths.  


Although being inducted into the Hall of Fame as a quarterback shouldn't be a criterion by which we evaluate a quarterback's passing ability, it does provide a reasonableness check of the method I've employed.  Given the likelihood of being in the Hall of Fame and showing up at the top of these lists, one could theoretically use it as a measure of whether a quarterback will be inducted into the Hall of Fame.  Obviously, there are other factors that enter that equation, but, I can draw some conclusions as to what might be necessary as a passer to be considered a good candidate for the Hall.


A C4 score of 370 or better is a must to be a shoo-in for the Hall.  The caveat there being the quarterback would have to had at least 8 seasons where they qualified.  Looking at C7, I'd say  a C7 score of 625 or better is a requirement, although a score of 600 or better conceivably also works.  A C10 score of 775 or better would be considered a must to have any chance of being inducted into the Hall of Fame (in my opinion anyway).


As I mentioned earlier in the article, I think C4 doesn't consider enough years.  Conversely, I think C10 is too exclusive.  C7 is a good compromise.  I could have created a C8 or C6, but C7 feels like it strikes the right balance between excellence and longevity.  From now on, I will use C7 as the standard measure of a player's career passing prowess, although I may use C4 and C10 where appropriate.  For example, if a player does not have a C7 score, I may use C4 as an early gauge, or, I may use C10 to compare to passers who have had lengthy careers.


Well, now that we've developed the method by which we can fairly present any quarterback's career as a passer, the natural question is, who is the greatest passer of all-time?  My method (C7) would suggest it is Joe Montana.  Steve Young would come in a close second.  You could also look at C4, and you'd see that they are the top 2, although in reverse order.  How good Joe Montana was, and for how long he performed at an incredibly high level, is evident by looking at C10 scores.  He is the only passer in the history of professional football to have a C10 score exceeding 900, and, at 939, he is 50 ahead of Peyton Manning.


A quick digression here, in case it wasn't obvious.  C scores can never decrease, only increase.  So for example, Peyton Manning's scores can only increase from here on out.  Since we consider a player's best years, if a subsequent year for a player isn't one of the best that already was included in the score, then it wouldn't count towards that score.  It would only count if it exceeded, and thereby replaced, one of the years already included.  Since Peyton's 10th best season so far is a 79 (his performance in 2007 was in the 79th percentile), he would have to have a performance better than that in the future to increase his C10 score (I would consider that quite likely).  On the other hand, his 7th best season is an 85 (his 2009 season), and I'd say that he's only about 50/50 to improve upon that, and thereby increase his C7 score (and if he did improve upon that, he would also improve his C10 score).


So, of the current crop of players, who might eventually knock on the door of the Hall of Fame?  Drew Brees and Tom Brady are currently ranked 21 and 22 on the C7 list with scores of 570 and 567, respectively.  A few more good years, and they would move up the ladder.  For example, in the case of Drew Brees, his score of 570 is made up of scores of 93, 93, 89, 88, 77, 75 and 55.  It's almost a guarantee that he would end up dropping the 55.  I think it's a foregone conclusion that his C7 score will eventually exceed 600 by a comfortable margin.  What about Brady?  Well, he is probably already a shoo-in for the Hall on account of his three Super Bowl victories.  But, as a passer, he appears a lot weaker than Brees.  Brady's C7 score of 567 is made up of scores of 98, 89, 80, 76, 75, 75, and 74.  Given his history, I don't see him improving too much from the 567.  Perhaps when it is all said and done, he might end up in the 580-590 range.  One never knows though.


What about the next generation of passers?  Who might be good candidates to have great careers as passers?  Chad Pennington, whose current C4 score is 366, one less than that of Kurt Warner, is an interesting case.  However, given his history of injuries, he may not be able to accumulate a solid C7 score.  If he has one decent year, say, in the 70th percentile, then his C7 score would be 575.  There are two players that I think could potentially have great careers based on their current C4 scores - Carson Palmer and Philip Rivers.  Based on their yearly scores, I think Philip Rivers might be the better bet, even though Palmer currently has the higher C4 score.  Interestingly, Philip Rivers currently ranks 2nd to Steve Young on the all-time passer rating list.  Of those who don't even have a C4 score, Matt Schaub has shown excellent early potential.

A Look Back At Kurt Warner's Career As a Passer

On the Friday before Superbowl XLIV, Kurt Warner announced that he was retiring from the NFL. 


Of course, the conversations now turn to whether he will be enshrined in Canton.  There are those who say yes, some who say maybe, and those who say no.  The fact is, we don't know.  There is no formula to get in. 


In many of the posts on the topic, the usual comparisons are made.  The easiest comparisons are to his peers - Manning, Brady, Favre, Brees, and others.  The other comparisons I've seen are to players like Dan Fouts, Jim Kelly, Troy Aikman, and Warren Moon, all four of whom are in Canton.


My post is not about whether he was a good quarterback or not, and whether he should or should not be in the Hall of Fame.  This post is about whether or not he was a good passer.  And how did he compare to his peers?  And where would he rank in terms of recent quarterbacks who have retired?  And how would he compare to all quarterbacks?  These are the questions I will explore.  All of these questions will be answered in the context of his passing ability in relation to the others.


Ok, let's start with a look back at his performance over the years.

Several things jump out at me.


His career passer rating, for one.  It's very high.  According to the Pro Football Hall of Fame, Warner's career passer rating of 93.7 ranks fifth best all-time, behind Steve Young, Philip Rivers, Tony Romo and Peyton Manning.  As I have discussed before, the NFL passer rating formula isn't a very meaningful way to compare quarterbacks over time.  I'll discuss how we adjust the passer rating, and even simplify it, and then compare that figure to his peers, in order to get a more meaningful picture.  The average passer rating of 95.1 shown above differs from that shown by the Pro Football Hall of Fame simply because mine is just a simple average of each year.


Another fact that jumps out at me is that he got a late start.  He didn't start really playing until he was 28.  In my database of 416 quarterbacks going all the way back to 1932, including players from both the AAFC in the late 1940's and the AFL in the 1960's, and only looking at quarterbacks who have qualified in 8 or more years, only 7 quarterbacks had their first qualifying season at age 28 or later.  For all quarterbacks who qualified in at least 8 seasons, both the median and mean age at which they qualified in their first season is 24.  There are a total of 77 quarterbacks who qualified in at least 8 seasons.  Keep in mind that the average number of years in which a quarterback qualifies (given that he qualifies in at least 1 year) is 4.125, and median number of years is 3.  So to be one of only 7 quarterbacks who began at age 28 or later and qualify in 8 different seasons is pretty rare company.  The other six players are Jeff Garcia and Roger Staubach at age 29, and Warren Moon, Billy Kilmer, Ken Stabler, and Brad Johnson at age 28.  Note that Staubach and Moon are both in the Hall of Fame.


Next thing I notice is that there are many gaps in his career.  He got injured, then was signed by the NY Giants as an insurance policy against Eli Manning.  Following that, he ended up getting an opportunity to play in Arizona, until the Cardinals drafted Matt Leinart.  When Leinart got hurt in the fifth game of the 2007 season, Kurt stepped in, and went on to lead the Cardinals into Superbowl XLIII, and into the playoffs this past season, where the Cardinals lost to eventual Superbowl Champs New Orleans.


It is extremely difficult to use the NFL Passer Rating system, especially the career passer rating for a given quarterback to see how they compare to other quarterbacks.  Just take a look at the list provided by the Pro Football Hall of Fame.  This is simply not a list of the best passers over time.  It merely reflects the fact that the passer rating calculation favors current passers.  I have previously shown (here) why this is not an accurate reflection of passing ability.


Perhaps one way to adjust the passer rating is to reflect the standards such that the average in each year turns out to be 66.7 (as the intended application of the passer rating).  See my post on the subject here.  In that regard, Warner's (adjusted) career passer rating is around 83.  Of the 416 quarterbacks, that would rank him 23rd.  Of the quarterbacks qualifying in 8 or more years, it would rank him 10th.


While the Adjusted Passer Rating is better than the NFL Passer Rating, in that at least one can compare quarterbacks from one era with those of another, it's still not an ideal (or simple) measure of a passer's effectiveness.  I've used CMTI in the past, and I'll use it here.  More specifically, I'll use the CMTIPR, or percentile rank of CMTI.


In my previous post, I compared Manning, Favre and Brees using CMTIPR.  We had to adjust CMTIPR for length of career, and I used their best seasons as the measure.  At the end of that post, a graphical illustration showed the effect.  I've since improved upon that concept.  Using the concept of "Best Seasons", I've come up with 4 statistical measures, C1, C4, C7 and C10, that basically shows a given quarterback's best season (C1), 4 best seasons (C4), 7 best seasons (C7) and 10 best seasons (C10).  C1 is simply the quarterback's best season in terms of percentile rank of CMTI.  C4 is similarly the sum of the percentile ranks of his 4 best seasons, and so on.


So let's take a look at Warner, and how he compares to Manning, Favre, and Brees.

Obviously, neither Brees nor Warner will have a C10 score as they've not had 10 seasons in which they qualified.  I don't think one can evaluate a career based on 1 year, so let's not use C1.  It is shown simply for illustration purposes to highlight their best year.  So that leaves us with C4 or C7.  In order to truly define greatness, one must exemplify superior performance, but also perform at that level for a number of years.  The more years performing at a high level, the greater the player.  I think that C7 strikes a good balance.  In my database, there are 50 quarterbacks who have a C10 score, 95 quarterbacks who have a C7 score, and 169 quarterbacks who have a C4 score.  So, at least based on a C7 score, Warner compares very favorably to Manning, Favre and Brees.


We will go into much more details regarding the C scores for quarterbacks in my next post.

Who's Better, Who's Best - Manning, Favre and Brees

Last Sunday we were all treated to some good football.  The AFC Championship featured the surprising NY Jets, against the Peyton Manning-led Indianapolis Colts, and the NFC Championship showcased the Drew Brees-led New Orleans Saints who played host to Brett Favre and his Minnesota Vikings.


Not only are these outstanding football teams, but three of the four teams featured three of the game's best quarterbacks in Manning, Favre and Brees.  I thought it would be illustrative to take a look at each of the three quarterbacks individually, and then compare the three side by side to get a sense who maybe better, and perhaps even, who may be best.  In evaluating their relative merits, I will only consider their ability to throw the football, not win games, or how well they run, or whether they are clutch performers, or how many Super Bowl victories they have.


Here are the career NFL Passer Ratings for these quarterbacks:

Manning is clearly the best.  Well, at least that's what looking at the career NFL Passer Rating will tell you.  These figures differ from what you would find at the Pro Football Hall of Fame (as of January 23rd, 2010).  The reason being that the figures listed on the Pro Football Hall of Fame site were as of the beginning of the 2009 season, and mine are as of the end of the 2009 season just completed.

For reasons that I have chronicled in the past (see here, here, and here), the NFL Passer Rating formula is in and of itself is of little use in determining who among the three might be best.  The three quarterbacks very well might end up being ranked this way when it is all said and done, but that doesn't justify the rating system.

In order to complete our analysis, we have to make several adjustments to the NFL's passer rating system.  

The first, is that we need to take each player's passer rating, and relate it to some baseline, so that we may account for players playing in different eras (yeah, I know, it's a little bit odd to be talking about players in different eras when comparing three current quarterbacks, but Brett Favre has been throwing since 1992, and Drew Brees only since 2002.  And since the average NFL passer rating has been increasing over time (see here), we need to make an adjustment for that.  The method that I have chosen is to relate a given quarterback's performance relative to the mean for that year, and use the standard deviation in that year as the measuring stick.  In other words, the difference between the quarterback's actual performance compared to the mean, as a function of the standard deviation.  As I described here, we then convert each of these resulting measures, also called z-values, into percentile ranks, using the standard normal distribution.

The second, is that we need to account for the fact that they've each played a different number of seasons.  While I used a rather simplistic approach last year, I will present a more sophisticated approach starting with this blog.  This method will need further refinement over time, since I am early in my thought process regarding this particular topic.  However, it is an improvement over last year's simplistic approach.  Actually, the method used last year (average deviation) really doesn't take into account the number of years at all.  When I evaluated Brett Favre's career, I simply compared him to a group of quarterbacks who had played a like number of years, and simply ignored any comparisons to players who played far fewer seasons.  That method won't work in this case, as we need to compare three quarterbacks with three vastly different career lengths.  All else being equal, and they rarely, if ever, are, a player who has a 10 year career versus a player who has a 7 year career should get a higher rating (even if, and especially if, they both had the exact same ratings in every year they played).  But the question becomes, "how much higher"?

The third, is that we need to somehow account for the fact that each of the three quarterbacks has played not only a different number of seasons, but that those seasons represent different parts of their careers.  In some respects, since the last 8 seasons have been common to all three, we could simply look at the last 8 seasons.  But, that would mean that we would be comparing Brett Favre in his 11th through 18th seasons, Peyton Manning through his 5th through 12th seasons, and Drew Brees his first 8 seasons.  As I have shown in the past, the early years for a quarterback tend to be the "learning years", and hence this comparison would be inappropriate without the proper adjustments.  Brett Favre is also 40 years old, and we have seen declining performances as quarterbacks age (actually, their performances increase at first as they gain experience, and then subsequently decline as the physical toll takes effect).  One method is to look at each player's first 8 years, since in that case the comparisons would be over the identical portions of their careers.  And perhaps we could compare Favre and Manning for the first 12 years.

Last, we need not assume that the NFL Passer Rating is the measure to be used to evaluate who might be the better passer.  The NFL Passer Rating system has a lot of flaws.  I will use two measurement methods.  The first is the concept of Adjusted Passer Rating.  As I have previously mentioned, the average NFL Passer Rating has been steadily increasing every year.  The Adjusted Passer Rating normalizes the passer rating so that the average for each year is 66.7.  This is then very helpful in evaluating players who played in different years (such as the case here).  Last year, I introduced a simple measure of passing efficiency - CMI.  I now call this CMTI (or, simplistically, Completions Minus Thrice Interceptions).  It is not only simple, but it has a 86% correlation with the NFL Passer Rating.  And, because it is so powerful in explaining any variances in the NFL Passer Rating, and because it is so easy to compute, use and explain, I will use it here.

So, to recap, we will make the following adjustments:

1. Relate performance in given year to the mean, and then in relation to standard deviation for that year - then convert to percentile rank
2. Use a 'better' measure than NFL Passer Rating.  We will show that Adjusted NFL Passer Rating is 'better', and CMTI is simpler
3. Take into account differing number of seasons played by each quarterback
4. Take into account quarterbacks who are at different stages of their career

Here are the respective #s for each quarterback by season:

Brett Favre

Drew Brees

Peyton Manning

What do these performance measures say?  Well, there's a lot of information here, so let's take these one at a time.


The first is that the career averages shown (actually all of the averages shown) are simple arithmetic averages of each of the years, not a true weighted average.  You can see that these simple averages are close enough to the true average by looking at the career average NFL passer ratings shown in these three exhibits and compare them to the career average for each of the quarterbacks shown earlier.


The second is that regardless of what measure we use, whether it is the NFL Passer Rating, Adjusted NFL Passer Rating, CMTI, first 8 years, last 12 years or what not, it should be very clear that Peyton Manning has performed at a truly superior level compared to the other 2 players (or anyone else for that matter).  Just look at CMTI over the past 8 years, for example.  He has averaged being in the 90th percentile over the past 8 years.  Basically, that suggests that he is, on average, in the Top 3 every year.  Truly remarkable.  In Peyton Manning, we are watching one of the all-time best.  A once-in-a-generation quarterback.  In case you didn't notice, the rows highlighted in green indicate a performance that is in the 85th percentile or higher.  The areas in blue are 95th percentile or higher.


Ok,  Manning is best.  That was easy.  Now comes the difficult part.  Who's better, Favre or Brees?


The case for Favre:


On an Adjusted Passer Rating basis, Brett Favre's first 8 years are comparable to, if not better than Drew Brees' first 8 years (70th percentile versus 68th percentile).  We can't directly compare their two careers since one has had an eighteen year career while the other only eight.  It is very evident that, but for the last year, Favre's performance has been quite pedestrian over the past 10 years.  Well, we can't necessarily penalize him for that, because we don't know how Brees might do in his next 10 years.  We don't even know if he is likely to play another 10 years.  Given history, however, it is fairly safe to assume that he won't play another 10 years.  In my database, going back to 1932, and including both the AAFC in the 1940's as well as the AFL in the 1960's, I have a total of 416 quarterbacks who qualified in at least 1 year.  206 of these quarterbacks qualified in only 1 or 2 years.  The average is 4.125 years.  There are only 6 quarterbacks that qualified in 15 or more years.  Y.A. Tittle qualified in fifteen different seasons.  John Elway, Dan Marino and Johnny Unitas each qualified in sixteen different years.  And then there are the ultimate marathon men, Fran Tarkenton and Brett Favre, who both qualified on 18 different occasions.


Michael J. Schell, in his book Baseball's All-Time Best Hitters, makes an adjustment to player batting averages, called a late-career adjustment.  In essence, he recognizes that a player's performance declines as he ages, and so he makes an adjustment for it.  Schell basically caps the at-bats of a player at 8,000.  All at-bats beyond 8,000 are ignored.  This way, he can compare each player through their first 8,000 at-bats.  Doing that type of adjustment would be extremely difficult in the case of football.  We know that the average number of passes attempted by a player per game has increased over time (see here and here), so limiting to a fixed number of attempts would be difficult and meaningless at best.


We could limit the player performances to a given number of seasons.  Look at the three graphs below.

The first is the average percentile rank of CMTI for all 416 quarterbacks by age.


And the second is the same as the first, except with the x-axis denoting the number of seasons in which they qualified instead of their age.


It appears that there is a learning curve at first (and an elimination process, as presumably the worst players end up having shorter careers), followed by a decline after about 12 years.  Keep in mind that this first graph shows the career percentile rank.  What we're really interested in is does performance decline as a player ages?  So, what we really want to see is the average percentile rank for a given year of a player's career no matter how long they have played.


This is what the third graph shows.  It's actually a bit surprising.  There doesn't appear to be a significant drop-off.  As a matter of fact, out in the 15th-18th years, their performance seems to have got better!  Ok, so in the outer years, we're not talking about a lot of players (for years 17 and 18 its just 2 players - Tarkenton and Favre), so anomalies should be expected.  So what do we do?


I mentioned earlier that the average number of years in which a quarterback qualifies is 4.125 for all 416 quarterbacks.  The table below shows the data in more detail.

There are a total of 77 quarterbacks who have played (qualified) in 8 or more years.  That would be 77/416 = 18.5% of the eligible list.  The average number of years for this group is 10.8, and the median number of years for this group is 11.  It would seem reasonable then to look at a minimum of 8 years as a cut-off.  But how do we recognize players who have played a lot longer?  The method that I have chosen is to look at a given player's best 8 years.  This means several things.  First, the obvious - it eliminates from consideration any player who has qualified in fewer than 8 years.  Second, it accounts for the fact that players have played longer, and, that fact doesn't hurt them.  As a matter of fact, it's the opposite.  The less a player has played, the more the "penalty".  Third, given the small sample of players playing beyond 14 years, it is quite possible that one or more of them had good years in those later years, so why exclude those years?


So, going back to the tables where I showed each of the three quarterbacks and their individual seasons, you can see that Favre's best 8 years are better than Brees' 8 years.  This gives credit to Favre's lengthy career.  Remember, less than 20% of the qualified quarterbacks even get to 8 years total, let alone be fortunate enough to have 8 good years.  Perhaps over time, Brees might very well exceed Favre.  And, if and when he does, we'll give him credit for it at that time.


The case for Brees:


Although comparable to Favre in his first 8 years based on Adjusted Passer Rating, using CMTI gives Brees the nod.  Also notice that Brees has had four years where he was better than 85th percentile, while Favre only had one.  Brees' last four years in New Orleans have been especially strong.  In each of years 5-8, Brees outperforms Favre, while in the first four, Favre outdoes Brees 3 out of four years.  Favre, it appears, built his legend largely on the success of his first year, and perhaps even starting with his first game.  Check out the following chart.

It shows the average percentile rank for all quarterbacks who played longer than 8 years, for each year they played.  Actually, it's not quite the number of years they played in as much as it is the number of years in which they threw enough passes to qualify to have a rating.  It's the blue line, and it includes data from the previously mentioned group of 77 quarterbacks.  The blue line in this graph is different than the line shown on the third graph earlier, in that this only includes data for those quarterbacks qualifying in 8 or more years.  The red line is the data for Drew Brees.  The green line is the data for Favre.  Clearly, outside of his first year, and then his 3 MVP years (years 3-5 for him), Favre hugged the average for this group of quarterbacks.  Some years he did better, and some he did worse.


We're almost done.  One more graph.

This one puts a slight twist on the notion of the best years.  Instead of just looking at the best 8 years, why don't we look at the best n years, for each year n that the quarterback has played.  So this way, we can look at the best year for each quarterback, the best two years, the best three years, and so on.  For good measure, I also threw in Manning's #s.


So what do we conclude from all of this?


Again, if there, was any doubt before, this should put it to rest.  Manning is clearly the best.


Between Favre and Brees, an argument could be made for either.  In terms of a total career, Favre gets the nod, simply because of the number of years he's played.  Brees has put up some impressive numbers so far.  If he continues this for a few more years, he will most likely finish ahead of Favre, but that remains to be seen.  Based on the evidence so far however, he is doing at least as well as, if not better than Favre did in his first 8 years.  Perhaps we'll check-in in a few years when Brees retires.  And, speaking of retirement, in my next post, I'll take a look at Kurt Warner's career.

The Adjusted NFL Passer Rating Revisited

This posting is an update and an enhancement to a previous posting that discussed the concept of the Adjusted NFL Passer Rating.  Back in 1971, when Don Smith, an executive with the Pro Football Hall of Fame, and his team developed the current NFL Passer Rating formula, he suggested that the intent of the formula was such that an 'average' Passer Rating would be equal to 66.7.  In essence, he decided that an average performance for each of the four components that make up the passer rating system should be equal to 1.0, with an exceptional performance defined as a score greater than 2.0, and a poor performance getting a 0.  Since the formula adds the scores for each of the four components and divides by 6, an average performance would be equal to 66.7.

Since we have seen that the average NFL passer rating has increased over time (see my previous post on the topic here), I was curious as to what the average in 2009 looked like, and, more importantly, what would the actual NFL passer ratings have looked like if we adjusted each of the components such that the averages for 2009 were in fact 1.0.

Recall that in the formula , based on data from the 1960's, adjustments are made to each of the four components such that the average for each component worked out to be 1.0.  The adjustments, call them k1, k2, k3 and k4, are:

• k1 = 30
• k2 = 3
• k3 = 1
• k4 = 1

Again, these factors were arrived at by looking at data from the 1960's.  And, these same factors are applied to the formula regardless of when a quarterback actually played.  This has the benefit of course of creating a formula that is the same for all quarterbacks against a fixed standard.  The problem of course is when the overall standard changes over time, and this is what has happened over the past 80 years or so.

Well, we don't have to use data from the 1960's.  We will use each year as a stand-alone year.  So, for 2009, the factors that make each component equal to 1.0 for an average performance turn out to be:

• 2009k1 = 40.90
• 2009k2 = 2.98
• 2009k3 = 1.20
• 2009k4 = 1.78

Likewise, the factors for 2008 were:

• 2008k1 = 41.00
• 2008k2 = 2.94
• 2008k3 = 1.28
• 2008k4 = 1.95

And, for good measure, the actual factors for the NFL for 1969 were:

• 1969k1 = 32.64
• 1969k2 = 2.99
• 1969k3 = 0.99
• 1969k4 = 1.12

These 1969 factors are very close to the k1, k2, k3 and k4 values used in the formula.

The table below shows the 2009 Adjusted Passer Ratings for the 32 quarterbacks that threw enough passes this past season to qualify.  The table also shows their actual 2009 NFL Passer Ratings.  You can see that the Adjusted Passer Ratings are clearly lower (as a group) than the actual NFL Passer Ratings.  That's because the actual NFL average passer rating in 2009 was 83.0, and, obviously, the average Adjusted Passer Rating for the NFL for 2009 was 66.7 (because we forced it to be so by adjusting the formula such that the average value for each component is equal to 1).

Although the Adjusted Passer Ratings themselves are a lot lower as a group than the actual NFL Passer Ratings, you'll notice that the rankings for each are essentially the same.  Why is that?  Very simply, since both calculations use the same essential formula with the values changed, all this really does is 'normalize' the actual ratings around a value of 66.7 as opposed to 83.0.  And yes, since the relative value of each of the four components now change, there should be some movement in the rankings.  Interestingly, I also included my CMTI calculations (and their respective ranks), and it also highly correlates with both the Adjusted Passer Rating rankings and the NFL Passer Rating Rankings.

Just so that you can see for yourself, I've included the table (shown below) that shows the average NFL passer rating by year from 1932 to 2009 using the current NFL passer rating formula (where k1 = 30, k2 = 3 and k3=k4=1 for all years), as well as the adjusted k1-k4 factors for each year.  There is no need to show the Adjusted Passer Rating for the NFL in each year, since it is the same in every year - 66.7.  If you look at the bottom of the table, for years 1960-1969, you'll see where Don Smith derived his k values from.

Looking at this table you should be able to understand why using the current NFL passer rating system is of little to no use when you need to compare quarterbacks from one era to another.  It can truly only be used to compare one passer versus another in any given year, but not across years.  It stands to reason then, that the current NFL passer rating formula isn't much use when evaluating a quarterback's career, especially ones who've had a long career (you can see the career NFL passer rating leaders here).  In future posts, I will discuss this in more detail.

Quarterbacks and the NFL Draft - Part 5

This post is the corollary to the previous post.  That is, we will attempt to answer the following question: which quarterbacks drafted in the past 30 years turned out to be the biggests busts?  Naturally, these would be the quarterbacks drafted with really high expectations - those that were drafted in the first round, or better yet, drafted in the Top 10, or even better, #1 overall, perhaps.  These cases of buyers remorse are further exacerbated when quarterbacks taken later (much later in some cases) turn out to have far superior careers.  

The list you see below, along with the list in my previous post, should illustrate the difficulty of identifying who might be the best NFL quarterbacks before their first game.  It is certainly an inexact science.  It is why I question the merits of a) picking a quarterback so early in the draft, and b) paying them such large sums of money before they ever throw a pass.  I will explore these questions in my next post.  

In the meantime, here's my list of the 10 quarterbacks who were the biggest busts of the NFL Draft over the past 30 years.  Unlike the biggest surprises, this list is a bit more difficult to assess.  For the surprises, one merely looks at the relevant statistics, and then finds an appropriate weight (for example, how much weight to assign to longevity).  For the busts, how does one quantify those that don't play (or don't play enough).  In my system of quantifying quarterback passing statistics, one must throw enough passes in a given season to qualify to be ranked (in other words, I don't rank every player that threw a pass as that would be meaningless).  For that reason, I shall merely list the players, and not rank them, starting with the earliest drafts.

The List

Jack Thompson (1979, 1st Round, 3rd Overall Pick, 1st Quarterback Drafted) - Thompson was drafted out of Washington State by the Cincinnati Bengals.  He was taken ahead of Phil Simms (7), and Joe Montana (82).  He qualified in 1 year.  He makes the list not because of his statistics as much as he was drafted so far in advance of the the quarterback who arguably was the greatest passer of all time.

Rich Campbell (1981, 1st Round, 6th Overall Pick, 1st Quarterback Drafted) - Campbell was drafted out of The University of California by the Green Bay Packers.  He was taken ahead of Neil Lomax (33).  Campbell didn't attempt enough passes in any year to qualify, and threw a total of 68 passes in two different seasons.

Art Schlichter (1982, 1st Round, 4th Overall Pick, 1st Quarterback Drafted) - Schlichter was drafted out of Ohio State by the Baltimore Colts.  The Chicago Bears selected Jim McMahon (5)  out of Brigham Young one pick later.  What bust list would be complete without Schlichter, known more famously for his gambling addictions.

Todd Blackledge (1983, 1st Round, 7th Overall Pick, 2nd Quarterback Drafted) - Blackledge was drafted out of Penn State by the Kansas City Chiefs.  Quarterbacks taken after Blackledge that year include Jim Kelly (14), Tony Eason (15), Ken O'Brien (24) and Dan Marino (27).  Todd qualified in 1984, a year when he completed 50.0% of his passes and had 3.7% of his attempts picked off.  John Elway was drafted 1st Overall that year.  Elway, Kelly and Marino are all in the Pro Football Hall of Fame.

Kelly Stouffer (1987, 1st Round, 6th Overall Pick, 2nd Quarterback Drafted) - Stouffer was drafted out of Colorado State by the St. Louis Cardinals.  Notable quarterbacks taken after Stouffer in that year's draft includes Jim Harbaugh (26), Rich Gannon (98) and Steve Beuerlein (110). Stouffer did not attempt enough passes in any given year to qualify, and threw a total of 437 passes over the course of four different seasons.

Heath Shuler (1994, 1st Round, 3rd Overall Pick, 1st Quarterback Drafted) - Shuler was drafted out of Tennessee by the Washington Redskins.  He qualified in 1 year (in 1994), when he completed 45.3% of his passes, and had 4.5% of his passes intercepted, a poor enough performance to be -2.54 standard deviations below the mean.

Ryan Leaf (1998, 1st Round, 2nd Overall Pick, 2nd Quarterback Drafted) - Leaf was almost picked by the Indianapolis Colts.  Instead, they went with some fellow named Peyton Manning.  The San Diego Chargers drafted Leaf out of Washington State.  Successful quarterbacks drafted following Leaf that year are Brian Griese (91) and Matt Hasselbeck (187).  In 2000, Leaf completed 50.0% of his passes, and had a 5.6% interception rate, a performance so poor it was -2.81 standard deviations below the mean.

Tim Couch (1999, 1st Round, 1st Overall Pick, 1st Quarterback Drafted) - Couch was drafted out of Kentucky by the Cleveland Browns.  That year, the following quarterbacks were also drafted.  Donovan McNabb (2), and Daunte Culpepper (11).  Couch played enough to qualify in 3 seasons with an average standard deviation of -0.58.  His best year was in 1999, when he finished when he completed 55.9% of his passes, and had 3.3% intercepted, good for -0.42 standard deviations below the mean.

Akili Smith (1999, 1st Round, 3rd Overall Pick, 3rd Quarterback Drafted) - Smith was drafted out of Oregon by the Cincinnati Bengals.  Donovan McNabb was taken 1 pick ahead of him, and Daunte Culpepper was taken 11th that year.  Smith qualified in 1 year (in 2000) when he completed just 44.2% of his passes and 2.2% picked off, sufficient to be -2.10 standard deviations below the mean.

Alex Smith (2005, 1st Round, 1st Overall Pick, 1st Quarterback Drafted) - Smith was drafted out of Utah by the San Francisco 49ers.  Although still a little early to be evaluating this draft, a couple of quarterbacks that were taken subsequent to Smith are Aaron Rodgers (24), and Jason  Campbell (25).  Smith has qualified in 1 year (2006), when he completed 58.1% of his passes and had 3.6% of his attempts intercepted.  He finished that year -0.61 standard deviations below the mean CMI.

That's a who's who of busts and disappointments.  I could name another dozen or so who arguably could make this list, but we must draw the line somewhere!

Quarterbacks and the NFL Draft - Part 4

In my last post, we looked at quarterbacks and their expected performance as compared to where they were drafted. We concluded that their on-field performance cannot be attributed to their draft position, and perhaps those drafted earlier play a bit longer. There definitely was a correlation between draft order, and their likelihood of actually playing in the NFL (playing as defined by throwing enough passes in a given year to qualify statistically).

All we really looked at in the previous post were averages over the past 30 years. We didn't spend any time looking at individual quarterbacks. In this post, we'll do so. We'll take a stab at answering the following question: Which quarterbacks have been the surprises, or the diamonds in the rough? These would be the quarterbacks that, when drafted, either had little likelihood of actually playing in the NFL, or, even when thought that they would play, surprised us as to how well they played and/or how long they played. These would also be quarterbacks that were drafted early in the draft (perhaps even in the first round), but worked out much better than the earlier picks.

Here is my list of the 10 best surprises of the NFL Draft over the past 30 years. Admittedly, it is somewhat subjective, but I do use objective criteria to determine the list.

10. Matt Hasselbeck (1998, 6th Round, 187th Overall Pick, 6th Quarterback Drafted) - Hasselbeck was drafted by the Green Bay Packers out of Boston College behind Peyton Manning (1), Ryan Leaf (2), Charlie Batch (60), Jonathan Quinn (86), and Brian Griese (91). Hasselbeck was traded by the Packers to the Seattle Seahawks in 2001. He has thrown enough passes to have qualified in 7 seasons, and his CMI has averaged 0.14 standard deviations above the mean over that period. His best year was in 2005, where he completed 65.5% of his passes and only had 2.0% of his pass attempts intercepted, a performance good enough to be 1.50 standard deviations above the mean.

9. Brian Griese (1998, 3rd Round, 91st Overall Pick, 5th Quarterback Drafted) - Griese was drafted by the Denver Broncos out of Michigan behind Peyton Manning (1), Ryan Leaf (2), Charlie Batch (60), and Jonathan Quinn (86). Griese became the starter in 1999 following the retirement of John Elway, and in 6 qualified seasons his CMI has averaged 0.42 standard deviations above the mean. His best season was 2000, where he completed 64.3% of his passes and only had 1.2% of his passes intercepted, worthy of being 2.04 standard deviations above the mean.

8. Neil O'Donnell (1990, 3rd Round, 70th Overall Pick, 5th Quarterback Drafted) - O'Donnell was drafted by the Pittsburgh Steelers out of the University of Maryland behind Jeff George (1), Andre Ware (7), Tom Hodson (59), and Peter Tom Willis (63). He started in 1991, qualified in 5 seasons for the Steelers, and threw enough passes in 2 more seasons - one for the Jets in 1997 and one for Cincinnati in 1998. He averaged 0.47 standard deviations better than the mean over those 7 years, and his best year was 1998, when he completed 61.8% of his passes, and was intercepted 1.2% of the time, good enough to finish 1.52 standard deviations above the mean.

7. Trent Green (1993, 8th Round, 222nd Overall Pick, 8th Quarterback Drafted) - Green was drafted by the San Diego Chargers out of Indiana University behind Drew Bledsoe (1), Rick Mirer (2), Billy Joe Hobert (58), Mark Brunell (118), Gino Torretta (192), Alex Van Pelt (216), and Elvis Grbac (219). He was the last quarterback drafted in 1993. He first threw enough passes to qualify in 1998 for the Washington Redskins. He qualified in 6 seasons altogether, the last 5 with the Kansas City Chiefs from 2001-2005. Over those 6 seasons, his CMI averaged 0.26 standard deviations above the mean. His best year came in 2003, when he completed 63.1% of his passes, and had 2.3% intercepted, good enough to finish 1.18 standard deviations better than the mean.

6. Mark Brunell (1993, 5th Round, 118th Overall Pick, 4th Quarterback Drafted) - Brunell was drafted by the Green Bay Packers out of the University of Washington behind Drew Bledsoe (1), Rick Mirer (2), and Billy Joe Hobert (58). He was traded to the Jacksonville Jaguars in 1995. He qualified in 8 straight seasons there, and then 2 more in 2005 and 2006 for the Redskins. Over those 10 seasons, his CMI averaged 0.64 standard deviations above the mean. His best year was 1997, where he completed 60.7% of his passes, and had 1.6% of his passes intercepted - good enough to be 1.23 standard deviations better than the mean.

5. Rich Gannon (1987, 4th Round, 98th Overall Pick, 7th Quarterback Drafted) - Gannon was drafted by the New England Patriots from the University of Delaware behind Vinny Testaverde (1), Kelly Stouffer (6), Chris Miller (13), Jim Harbaugh (26), Cody Carlson (64), and Mark Vlasic (88). He was traded to the Minnesota Vikings and qualified in three years (1990-1992) while there. He threw enough passes to qualify again in 1998 with Kansas City, and then from 1999-2002 with the Oakland Raiders. Over the 10 qualifying seasons, he averaged 0.64 standard deviations better than the mean. His best year was in 2001, when he completed 65.8% of his passes, and only had 1.6% of his attempts intercepted. This was good enough to be 2.54 standard deviations better than the mean that year.

4. Dan Marino (1983, 1st Round, 27th Overall Pick, 6th Quarterback Drafted) - This was the "Year of the Quarterback". Marino was drafted by the Miami Dolphins out of the University of Pittsburgh behind John Elway (1), Todd Blackledge (7), Jim Kelly (14), Tony Eason (15), and Ken O'Brien (24). He started immediately, and went on to throw enough passes in 16 different seasons, averaging 0.51 standard deviations above the mean CMI. His best year was 1984, where he completed 64.2% of his passes, and had 3.0% of his passes intercepted. This was good to finish 1.28 standard deviations above the mean CMI that year. Dan Marino was elected to the Pro Football Hall of Fame in 1995.

3. Brad Johnson (1992, 9th Round, 227th Overall Pick, 13th Quarterback Drafted) - Johnson was drafted by the Minnesota Vikings out of Florida State behind David Klingler (6), Tommy Maddox (25), Matt Blundin (40), Tony Sacca (46), Craig Erickson (86), Casey Weldon (102), Will Furer (107), Chris Hakel (112), Jeff Blake (166), Kent Graham (211), Bucky Richardson (220), and Mike Pawlawski (222). He qualified in 9 different seasons - first with Minnesota (1996-1997), then the Redskins (1999-2000), then Tampa Bay (2001-2003), and finally with the Vikings again (2005-2006). Over those 9 seasons, his CMI averaged 0.73 standard deviations better than the mean. 2005 was his best year, where he completed 62.6% of his passes and only 1.4% of his attempts were intercepted, a performance good enough to be 1.32 standard deviations better than the mean CMI that year.

2. Tom Brady (2000, 6th Round, 199th Overall Pick, 7th Quarterback Drafted) - Brady was selected by the New England Patriots from the University of Michigan behind Chad Pennington (18), Giovanni Carmazzi (65), Chris Redman (75), Tee Martin (163), Marc Bulger (168), and Spergon Wynn (183). After replacing an injured Drew Bledsoe in the second game of the 2001 season, he started every game until he got injured during the first game of the 2008 season. Over the 7 year period from 2001-2007, his CMI was good enough to average 0.82 standard deviations above the mean. His best year came in 2007, when he completed 68.9% of his passes, and was intercepted 1.4% of the time, good enough for 2.14 standard deviations above the mean.  Tom Brady will likely get into the Hall of Fame.

1. Joe Montana (1979, 3rd Round, 82nd Overall Pick, 4th Quarterback Drafted) - Montana was drafted by the San Francisco Forty Niners from the University of Notre Dame behind Jack Thompson (3), Phil Simms (7), and Steve Fuller (23). Although he played every single game in the 1979 season, he only threw 23 passes that year. Beginning in 1980, he qualified every year until 1990, when injuries slowed him down. He was eventually traded to the Kansas City Chiefs, where he played for 2 more seasons. Over those 13 years, his performances on the field, he averaged 1.54 standard deviations above the mean, including the remarkable feat of never having a season below the mean. His best season was 1989, when he completed 70.2% of his passes and had 2.1% of his pass attempts intercepted, a performance so good relative to his peers, that it ranked 2.96 standard deviations above the mean.  Montana was elected to the Pro Football Hall of Fame in 2000.

Now that's a pretty good list!