C Scores - A New Way to Evaluate Pro Football's Best Passers of All-Time

It has taken me two years, many, many iterations, and countless hours of mental anguish, but I think I've finally settled on a reliable method by which I can evaluate any quarterback's passing ability versus any other quarterback's passing ability regardless of when they played, or for how long they played.  So, for example, I can compare Peyton Manning versus Sammy Baugh, or, Otto Graham versus Brett Favre, or, Sonny Jurgensen versus Dan Marino.  You get the point.  It all started coming together for me as I wrote my post on Manning versus Brees versus Favre.  While I had already figured out how to effectively compare one quarterback's given season to that of another's given season, I had as yet to figure out an effective method by which we evaluated careers.

The inspiration came to me as I read Michael J. Schell's book, Baseball's All-Time Best Hitters, where he went about solving a different problem altogether, but had essentially the same characteristics of the problem I was trying to solve.  He was trying to account for the fact that for many hitters with long careers, their batting averages declined after they reached their mid-thirties.  And, he didn't think it was fair to compare a hitter who had played into their late thirties or early forties to a player that had not.  He chose to account for this by using a "late career adjustment".  What he did was to look only at a player's first 8,000 at-bats.  


I adopted this notion, and, instead of using a player's first x years, came up with the idea of using his best x years, given that a player's passing ability, as measured by CMTI, and compared to that of his peers tends to be a little volatile from year to year.


Deciding what x was going to be was an issue.  I wanted to balance outstanding achievement with duration.  In other words, I wanted to see who had performed at an exceedingly high level for a long time.  Remember, from my previous post, the average number of years a quarterback qualifies is 4.125.  The median is 3.  


I thought that a good starting point was to use 4 years.  This measure would include 169 players, or a little more than 40% of my database of 416 players going back to when the NFL officially started keeping statistics, and including both the AAFC in the 1940's and the AFL in the 1960's.


If I used 7 years, then I'd get 95 players, or almost 25% of the total.  That seemed reasonable for a couple of reasons.  First, 25% seemed like a good cut-off, albeit arbitrary.  Second, it would allow me to discard the worst years for those quarterbacks who had 8 or more years, which seemed like an appropriate way to reward longevity.  


I could have looked at 10 years - in which case, we would be limiting ourselves to 50 quarterbacks, or about 12% of the database, a rather small group, and, as we shall see, excluding many superb passers who didn't quite make it to 10 years.  Keep in mind that the median number of years a player qualifies, is 3 years.


My basic method for evaluating a player's performance in a given year is to look at his CMTI, and then, based on the mean and standard deviation for that given year, relate his performance to that of his peers.  Since the aggregate accumulation of these measures over the past 78 years looks very much like a normal (bell-shaped) curve, we can use the bell-curve to convert these measures to percentile ranks, giving a numerical value ranging from 1 to 99 for each of these performances.  The sum of a given quarterback's 4 best such performances are then used as my C4 calculation, and the sum of his 7 best performances are my C7 calculation.  Obviously, then the maximum possible value for C4 is 99 * 4 = 396, and, commensurately, the maximum C7 value is 99 * 7 = 693.  As I mentioned earlier, I've also calculated a C10 value for those players with an exceptionally long career.  Obviously, if we did this for every year a player qualified, one could create a career C score for each player.


Let's take a look at the lists.  We will start with C4.

Wow!  What a list.  Not bad for starters.  Remember, this is a ranking based on a quarterback's 4 best years as a passer.  Look at the top 14 on this list.  12 of them are in the Pro Football Hall of Fame!  Peyton Manning surely will be when he is done.  That's 13 out of 14!  Ken Anderson ranks 3rd all-time as a passer according to this list.  It is important to keep in mind that this is a list of the top-ranked passers, not top-ranked quarterbacks.  It is hard enought to determine a ranking of passers, so it would stand to reason then that it is much harder to determine what makes a top-ranked quarterback.  I don't know how the NFL goes about determining who gets in to the Hall of Fame and who deos not, but it sure seems odd that Ken Anderson is the only player in the top 14 on this list that isn't (assuming that Peyton is in, of course). 

One of the most aesthetically pleasing aspects of this list, especially looking at the top 15, is that every era and every decade is represented.  There does not seem to be a bias.  I would think that is an admirable quality of any top-ranked list in that it shows that the measure used to determine who is better, isn't necessarily biased towards a particular era - as opposed to say, the list of top ranked passers (see here and here) in terms of the NFL Passer Rating system at the Pro-Football Hall of Fame.  


Look at Steve Young's C4.  That's almost unbelievable!  The highest C4 possible is 396.  In other words, that is being in the 99th percentile four years.  Well, Young comes close.  His 1996 and 1997 seasons were in the 99th percentile, and his 1992 and 1994 seasons were in the 98th percentile.


In my previous post, we discussed Kurt Warner's merits as a passer.  We ignored any discussion about whether he will be enshrined in Canton.  Based on this list, it would appear that he is right on the fence.


There are a total of 27 quarterbacks on this list that are members of the Hall of Fame.  There are a total of 31 quarterbacks that have been inducted into the Hall of Fame.  The four that are in the Hall of Fame, but are not on this list are omitted for good reason.  They all played in the 'early years' of the NFL - prior to the keeping of official statistics.  They are, Jimmy Conzelman, who played from 1921-1929, John (Paddy) Driscoll, who played from 1920-1929, Earl (Dutch) Clark, who played from 1931-1938 and actually had two very good years in 1934 and 1936, and Benny Friedman, who played from 1927-1934, and who, many during his time argued was the finest passer of his time.  It is most unfortunate that we don't have official statistics that would show that.


OK, what about C7?

Woah!  An even better list than C4!  First, while the ordering changed, the top 15 stayed exactly the same as the C4 list, which probably suggests more of the power of the C4 list than anything else (in other words, a quarterback's best 4 years is a pretty good indicator of their passing ability).  However, now, we not only have 13 of the top 15 in the HOF (I am considering it a foregone conclusion that Peyton Manning will be inducted into the HOF), but 16 of the top 20 are either in the Hall of Fame or will be (I am also considering it a foregone conclusion that Brett Favre will also be inducted into the HOF).  That is a phenomenal list.  


It seems obvious then, that, in order to be inducted into the Hall of Fame as a quarterback, one must also have been a great passer.  Having said that, it's even more shocking to me now the omission of Ken Anderson from the HOF.  I simply do not understand why he is not in.  I will have to post an article on Ken Anderson in the near future.


What does C7 say about Kurt Warner?  Again, right on the fence.  

From my vantage point, it certainly appears that we should be discussing putting Ken Anderson in the Hall of Fame ahead of Kurt Warner.


Finally, we'll take a look at C10.  This list only includes 50 quarterbacks, and many exceptional passers are not on this list, simply because they didn't qualify in at least 10 years.

The story is similar to what we've already seen with the C4 and C7 lists.  14 of the top 15 (if you include Peyton Manning and Brett Favre as in the Hall of Fame) are or will be in the Hall of Fame.  The lone exception is - you guessed it - Ken Anderson.


There's several conclusions that I can draw from all of this.


I feel pretty good that I've come up with a method by which I can determine the all-time best passers in the history of professional football.  I also feel that this method will stand the test of time.  It will work regardless of how the game changes over time.  It is a method that allows me to conveniently compare players who played in different eras, and it allows me to compare players who have had different career lengths.  


Although being inducted into the Hall of Fame as a quarterback shouldn't be a criterion by which we evaluate a quarterback's passing ability, it does provide a reasonableness check of the method I've employed.  Given the likelihood of being in the Hall of Fame and showing up at the top of these lists, one could theoretically use it as a measure of whether a quarterback will be inducted into the Hall of Fame.  Obviously, there are other factors that enter that equation, but, I can draw some conclusions as to what might be necessary as a passer to be considered a good candidate for the Hall.


A C4 score of 370 or better is a must to be a shoo-in for the Hall.  The caveat there being the quarterback would have to had at least 8 seasons where they qualified.  Looking at C7, I'd say  a C7 score of 625 or better is a requirement, although a score of 600 or better conceivably also works.  A C10 score of 775 or better would be considered a must to have any chance of being inducted into the Hall of Fame (in my opinion anyway).


As I mentioned earlier in the article, I think C4 doesn't consider enough years.  Conversely, I think C10 is too exclusive.  C7 is a good compromise.  I could have created a C8 or C6, but C7 feels like it strikes the right balance between excellence and longevity.  From now on, I will use C7 as the standard measure of a player's career passing prowess, although I may use C4 and C10 where appropriate.  For example, if a player does not have a C7 score, I may use C4 as an early gauge, or, I may use C10 to compare to passers who have had lengthy careers.


Well, now that we've developed the method by which we can fairly present any quarterback's career as a passer, the natural question is, who is the greatest passer of all-time?  My method (C7) would suggest it is Joe Montana.  Steve Young would come in a close second.  You could also look at C4, and you'd see that they are the top 2, although in reverse order.  How good Joe Montana was, and for how long he performed at an incredibly high level, is evident by looking at C10 scores.  He is the only passer in the history of professional football to have a C10 score exceeding 900, and, at 939, he is 50 ahead of Peyton Manning.


A quick digression here, in case it wasn't obvious.  C scores can never decrease, only increase.  So for example, Peyton Manning's scores can only increase from here on out.  Since we consider a player's best years, if a subsequent year for a player isn't one of the best that already was included in the score, then it wouldn't count towards that score.  It would only count if it exceeded, and thereby replaced, one of the years already included.  Since Peyton's 10th best season so far is a 79 (his performance in 2007 was in the 79th percentile), he would have to have a performance better than that in the future to increase his C10 score (I would consider that quite likely).  On the other hand, his 7th best season is an 85 (his 2009 season), and I'd say that he's only about 50/50 to improve upon that, and thereby increase his C7 score (and if he did improve upon that, he would also improve his C10 score).


So, of the current crop of players, who might eventually knock on the door of the Hall of Fame?  Drew Brees and Tom Brady are currently ranked 21 and 22 on the C7 list with scores of 570 and 567, respectively.  A few more good years, and they would move up the ladder.  For example, in the case of Drew Brees, his score of 570 is made up of scores of 93, 93, 89, 88, 77, 75 and 55.  It's almost a guarantee that he would end up dropping the 55.  I think it's a foregone conclusion that his C7 score will eventually exceed 600 by a comfortable margin.  What about Brady?  Well, he is probably already a shoo-in for the Hall on account of his three Super Bowl victories.  But, as a passer, he appears a lot weaker than Brees.  Brady's C7 score of 567 is made up of scores of 98, 89, 80, 76, 75, 75, and 74.  Given his history, I don't see him improving too much from the 567.  Perhaps when it is all said and done, he might end up in the 580-590 range.  One never knows though.


What about the next generation of passers?  Who might be good candidates to have great careers as passers?  Chad Pennington, whose current C4 score is 366, one less than that of Kurt Warner, is an interesting case.  However, given his history of injuries, he may not be able to accumulate a solid C7 score.  If he has one decent year, say, in the 70th percentile, then his C7 score would be 575.  There are two players that I think could potentially have great careers based on their current C4 scores - Carson Palmer and Philip Rivers.  Based on their yearly scores, I think Philip Rivers might be the better bet, even though Palmer currently has the higher C4 score.  Interestingly, Philip Rivers currently ranks 2nd to Steve Young on the all-time passer rating list.  Of those who don't even have a C4 score, Matt Schaub has shown excellent early potential.