Pro Football's All-Time Best Passers

Now that I've got a method for evaluating a quarterback as a passer and can use it compare his career to that of others, I thought it might be illustrative to compare the Top 50 on my list to how they rank on the all-time passer rating list provided by the Pro Football Hall of Fame.  The list provided by the Hall only includes those passers who've attempted 1,500 or more passes in their career.  Also, as of today (April 27, 2010), the Hall of Fame list does not include stats from the 2009 season.  As I've mentioned in the past, the HOF list unfairly penalizes those players who played in an era where passing was secondary.  Why?  Because of the restriction of limiting it to only those passers attempting 1,500 or more passes.  So, to supplement the list from the HOF, I'll show you their passer rating ranking based on my list of 416 passers, which is a broader and more inclusive list, especially of those who played in the 1930's - 1970's.  My list includes any quarterback who qualified in at least one season.

Here's the top 50 list of All-Time Best Passers:

My list of 50 not only covers the entire spectrum of pro football history - from the beginning in the 1930's to current players, but it also presents the players who played in earlier years fairly, and in context.  The question isn't whether Joe Montana was a better passer than Sammy Baugh, but, whether Joe Montana relative to Montana's peers, was a better passer than Sammy Baugh was relative to Baugh's peers.  The HOF list can be used to answer that question, since a measure relative to a fixed standard (in other words the passer rating system) can be used to do that.  However, the HOF list cannot be used to answer the next question - how does one compare Sammy Baugh to Joe Montana given that they both played an entirely different game, with different rules and circumstances.  A fixed standard cannot solve that particular puzzle.  The use of a mean and standard deviation by year (z scores) which in turn we convert to percentile ranks can be used.  C7 is simply a quarterback's 7 best years as a passer in terms of percentile rank summed up.

C Scores - A New Way to Evaluate Pro Football's Best Passers of All-Time

It has taken me two years, many, many iterations, and countless hours of mental anguish, but I think I've finally settled on a reliable method by which I can evaluate any quarterback's passing ability versus any other quarterback's passing ability regardless of when they played, or for how long they played.  So, for example, I can compare Peyton Manning versus Sammy Baugh, or, Otto Graham versus Brett Favre, or, Sonny Jurgensen versus Dan Marino.  You get the point.  It all started coming together for me as I wrote my post on Manning versus Brees versus Favre.  While I had already figured out how to effectively compare one quarterback's given season to that of another's given season, I had as yet to figure out an effective method by which we evaluated careers.

The inspiration came to me as I read Michael J. Schell's book, Baseball's All-Time Best Hitters, where he went about solving a different problem altogether, but had essentially the same characteristics of the problem I was trying to solve.  He was trying to account for the fact that for many hitters with long careers, their batting averages declined after they reached their mid-thirties.  And, he didn't think it was fair to compare a hitter who had played into their late thirties or early forties to a player that had not.  He chose to account for this by using a "late career adjustment".  What he did was to look only at a player's first 8,000 at-bats.  


I adopted this notion, and, instead of using a player's first x years, came up with the idea of using his best x years, given that a player's passing ability, as measured by CMTI, and compared to that of his peers tends to be a little volatile from year to year.


Deciding what x was going to be was an issue.  I wanted to balance outstanding achievement with duration.  In other words, I wanted to see who had performed at an exceedingly high level for a long time.  Remember, from my previous post, the average number of years a quarterback qualifies is 4.125.  The median is 3.  


I thought that a good starting point was to use 4 years.  This measure would include 169 players, or a little more than 40% of my database of 416 players going back to when the NFL officially started keeping statistics, and including both the AAFC in the 1940's and the AFL in the 1960's.


If I used 7 years, then I'd get 95 players, or almost 25% of the total.  That seemed reasonable for a couple of reasons.  First, 25% seemed like a good cut-off, albeit arbitrary.  Second, it would allow me to discard the worst years for those quarterbacks who had 8 or more years, which seemed like an appropriate way to reward longevity.  


I could have looked at 10 years - in which case, we would be limiting ourselves to 50 quarterbacks, or about 12% of the database, a rather small group, and, as we shall see, excluding many superb passers who didn't quite make it to 10 years.  Keep in mind that the median number of years a player qualifies, is 3 years.


My basic method for evaluating a player's performance in a given year is to look at his CMTI, and then, based on the mean and standard deviation for that given year, relate his performance to that of his peers.  Since the aggregate accumulation of these measures over the past 78 years looks very much like a normal (bell-shaped) curve, we can use the bell-curve to convert these measures to percentile ranks, giving a numerical value ranging from 1 to 99 for each of these performances.  The sum of a given quarterback's 4 best such performances are then used as my C4 calculation, and the sum of his 7 best performances are my C7 calculation.  Obviously, then the maximum possible value for C4 is 99 * 4 = 396, and, commensurately, the maximum C7 value is 99 * 7 = 693.  As I mentioned earlier, I've also calculated a C10 value for those players with an exceptionally long career.  Obviously, if we did this for every year a player qualified, one could create a career C score for each player.


Let's take a look at the lists.  We will start with C4.

Wow!  What a list.  Not bad for starters.  Remember, this is a ranking based on a quarterback's 4 best years as a passer.  Look at the top 14 on this list.  12 of them are in the Pro Football Hall of Fame!  Peyton Manning surely will be when he is done.  That's 13 out of 14!  Ken Anderson ranks 3rd all-time as a passer according to this list.  It is important to keep in mind that this is a list of the top-ranked passers, not top-ranked quarterbacks.  It is hard enought to determine a ranking of passers, so it would stand to reason then that it is much harder to determine what makes a top-ranked quarterback.  I don't know how the NFL goes about determining who gets in to the Hall of Fame and who deos not, but it sure seems odd that Ken Anderson is the only player in the top 14 on this list that isn't (assuming that Peyton is in, of course). 

One of the most aesthetically pleasing aspects of this list, especially looking at the top 15, is that every era and every decade is represented.  There does not seem to be a bias.  I would think that is an admirable quality of any top-ranked list in that it shows that the measure used to determine who is better, isn't necessarily biased towards a particular era - as opposed to say, the list of top ranked passers (see here and here) in terms of the NFL Passer Rating system at the Pro-Football Hall of Fame.  


Look at Steve Young's C4.  That's almost unbelievable!  The highest C4 possible is 396.  In other words, that is being in the 99th percentile four years.  Well, Young comes close.  His 1996 and 1997 seasons were in the 99th percentile, and his 1992 and 1994 seasons were in the 98th percentile.


In my previous post, we discussed Kurt Warner's merits as a passer.  We ignored any discussion about whether he will be enshrined in Canton.  Based on this list, it would appear that he is right on the fence.


There are a total of 27 quarterbacks on this list that are members of the Hall of Fame.  There are a total of 31 quarterbacks that have been inducted into the Hall of Fame.  The four that are in the Hall of Fame, but are not on this list are omitted for good reason.  They all played in the 'early years' of the NFL - prior to the keeping of official statistics.  They are, Jimmy Conzelman, who played from 1921-1929, John (Paddy) Driscoll, who played from 1920-1929, Earl (Dutch) Clark, who played from 1931-1938 and actually had two very good years in 1934 and 1936, and Benny Friedman, who played from 1927-1934, and who, many during his time argued was the finest passer of his time.  It is most unfortunate that we don't have official statistics that would show that.


OK, what about C7?

Woah!  An even better list than C4!  First, while the ordering changed, the top 15 stayed exactly the same as the C4 list, which probably suggests more of the power of the C4 list than anything else (in other words, a quarterback's best 4 years is a pretty good indicator of their passing ability).  However, now, we not only have 13 of the top 15 in the HOF (I am considering it a foregone conclusion that Peyton Manning will be inducted into the HOF), but 16 of the top 20 are either in the Hall of Fame or will be (I am also considering it a foregone conclusion that Brett Favre will also be inducted into the HOF).  That is a phenomenal list.  


It seems obvious then, that, in order to be inducted into the Hall of Fame as a quarterback, one must also have been a great passer.  Having said that, it's even more shocking to me now the omission of Ken Anderson from the HOF.  I simply do not understand why he is not in.  I will have to post an article on Ken Anderson in the near future.


What does C7 say about Kurt Warner?  Again, right on the fence.  

From my vantage point, it certainly appears that we should be discussing putting Ken Anderson in the Hall of Fame ahead of Kurt Warner.


Finally, we'll take a look at C10.  This list only includes 50 quarterbacks, and many exceptional passers are not on this list, simply because they didn't qualify in at least 10 years.

The story is similar to what we've already seen with the C4 and C7 lists.  14 of the top 15 (if you include Peyton Manning and Brett Favre as in the Hall of Fame) are or will be in the Hall of Fame.  The lone exception is - you guessed it - Ken Anderson.


There's several conclusions that I can draw from all of this.


I feel pretty good that I've come up with a method by which I can determine the all-time best passers in the history of professional football.  I also feel that this method will stand the test of time.  It will work regardless of how the game changes over time.  It is a method that allows me to conveniently compare players who played in different eras, and it allows me to compare players who have had different career lengths.  


Although being inducted into the Hall of Fame as a quarterback shouldn't be a criterion by which we evaluate a quarterback's passing ability, it does provide a reasonableness check of the method I've employed.  Given the likelihood of being in the Hall of Fame and showing up at the top of these lists, one could theoretically use it as a measure of whether a quarterback will be inducted into the Hall of Fame.  Obviously, there are other factors that enter that equation, but, I can draw some conclusions as to what might be necessary as a passer to be considered a good candidate for the Hall.


A C4 score of 370 or better is a must to be a shoo-in for the Hall.  The caveat there being the quarterback would have to had at least 8 seasons where they qualified.  Looking at C7, I'd say  a C7 score of 625 or better is a requirement, although a score of 600 or better conceivably also works.  A C10 score of 775 or better would be considered a must to have any chance of being inducted into the Hall of Fame (in my opinion anyway).


As I mentioned earlier in the article, I think C4 doesn't consider enough years.  Conversely, I think C10 is too exclusive.  C7 is a good compromise.  I could have created a C8 or C6, but C7 feels like it strikes the right balance between excellence and longevity.  From now on, I will use C7 as the standard measure of a player's career passing prowess, although I may use C4 and C10 where appropriate.  For example, if a player does not have a C7 score, I may use C4 as an early gauge, or, I may use C10 to compare to passers who have had lengthy careers.


Well, now that we've developed the method by which we can fairly present any quarterback's career as a passer, the natural question is, who is the greatest passer of all-time?  My method (C7) would suggest it is Joe Montana.  Steve Young would come in a close second.  You could also look at C4, and you'd see that they are the top 2, although in reverse order.  How good Joe Montana was, and for how long he performed at an incredibly high level, is evident by looking at C10 scores.  He is the only passer in the history of professional football to have a C10 score exceeding 900, and, at 939, he is 50 ahead of Peyton Manning.


A quick digression here, in case it wasn't obvious.  C scores can never decrease, only increase.  So for example, Peyton Manning's scores can only increase from here on out.  Since we consider a player's best years, if a subsequent year for a player isn't one of the best that already was included in the score, then it wouldn't count towards that score.  It would only count if it exceeded, and thereby replaced, one of the years already included.  Since Peyton's 10th best season so far is a 79 (his performance in 2007 was in the 79th percentile), he would have to have a performance better than that in the future to increase his C10 score (I would consider that quite likely).  On the other hand, his 7th best season is an 85 (his 2009 season), and I'd say that he's only about 50/50 to improve upon that, and thereby increase his C7 score (and if he did improve upon that, he would also improve his C10 score).


So, of the current crop of players, who might eventually knock on the door of the Hall of Fame?  Drew Brees and Tom Brady are currently ranked 21 and 22 on the C7 list with scores of 570 and 567, respectively.  A few more good years, and they would move up the ladder.  For example, in the case of Drew Brees, his score of 570 is made up of scores of 93, 93, 89, 88, 77, 75 and 55.  It's almost a guarantee that he would end up dropping the 55.  I think it's a foregone conclusion that his C7 score will eventually exceed 600 by a comfortable margin.  What about Brady?  Well, he is probably already a shoo-in for the Hall on account of his three Super Bowl victories.  But, as a passer, he appears a lot weaker than Brees.  Brady's C7 score of 567 is made up of scores of 98, 89, 80, 76, 75, 75, and 74.  Given his history, I don't see him improving too much from the 567.  Perhaps when it is all said and done, he might end up in the 580-590 range.  One never knows though.


What about the next generation of passers?  Who might be good candidates to have great careers as passers?  Chad Pennington, whose current C4 score is 366, one less than that of Kurt Warner, is an interesting case.  However, given his history of injuries, he may not be able to accumulate a solid C7 score.  If he has one decent year, say, in the 70th percentile, then his C7 score would be 575.  There are two players that I think could potentially have great careers based on their current C4 scores - Carson Palmer and Philip Rivers.  Based on their yearly scores, I think Philip Rivers might be the better bet, even though Palmer currently has the higher C4 score.  Interestingly, Philip Rivers currently ranks 2nd to Steve Young on the all-time passer rating list.  Of those who don't even have a C4 score, Matt Schaub has shown excellent early potential.

Quarterbacks and the NFL Draft - Part 5

This post is the corollary to the previous post.  That is, we will attempt to answer the following question: which quarterbacks drafted in the past 30 years turned out to be the biggests busts?  Naturally, these would be the quarterbacks drafted with really high expectations - those that were drafted in the first round, or better yet, drafted in the Top 10, or even better, #1 overall, perhaps.  These cases of buyers remorse are further exacerbated when quarterbacks taken later (much later in some cases) turn out to have far superior careers.  

The list you see below, along with the list in my previous post, should illustrate the difficulty of identifying who might be the best NFL quarterbacks before their first game.  It is certainly an inexact science.  It is why I question the merits of a) picking a quarterback so early in the draft, and b) paying them such large sums of money before they ever throw a pass.  I will explore these questions in my next post.  

In the meantime, here's my list of the 10 quarterbacks who were the biggest busts of the NFL Draft over the past 30 years.  Unlike the biggest surprises, this list is a bit more difficult to assess.  For the surprises, one merely looks at the relevant statistics, and then finds an appropriate weight (for example, how much weight to assign to longevity).  For the busts, how does one quantify those that don't play (or don't play enough).  In my system of quantifying quarterback passing statistics, one must throw enough passes in a given season to qualify to be ranked (in other words, I don't rank every player that threw a pass as that would be meaningless).  For that reason, I shall merely list the players, and not rank them, starting with the earliest drafts.

The List

Jack Thompson (1979, 1st Round, 3rd Overall Pick, 1st Quarterback Drafted) - Thompson was drafted out of Washington State by the Cincinnati Bengals.  He was taken ahead of Phil Simms (7), and Joe Montana (82).  He qualified in 1 year.  He makes the list not because of his statistics as much as he was drafted so far in advance of the the quarterback who arguably was the greatest passer of all time.

Rich Campbell (1981, 1st Round, 6th Overall Pick, 1st Quarterback Drafted) - Campbell was drafted out of The University of California by the Green Bay Packers.  He was taken ahead of Neil Lomax (33).  Campbell didn't attempt enough passes in any year to qualify, and threw a total of 68 passes in two different seasons.

Art Schlichter (1982, 1st Round, 4th Overall Pick, 1st Quarterback Drafted) - Schlichter was drafted out of Ohio State by the Baltimore Colts.  The Chicago Bears selected Jim McMahon (5)  out of Brigham Young one pick later.  What bust list would be complete without Schlichter, known more famously for his gambling addictions.

Todd Blackledge (1983, 1st Round, 7th Overall Pick, 2nd Quarterback Drafted) - Blackledge was drafted out of Penn State by the Kansas City Chiefs.  Quarterbacks taken after Blackledge that year include Jim Kelly (14), Tony Eason (15), Ken O'Brien (24) and Dan Marino (27).  Todd qualified in 1984, a year when he completed 50.0% of his passes and had 3.7% of his attempts picked off.  John Elway was drafted 1st Overall that year.  Elway, Kelly and Marino are all in the Pro Football Hall of Fame.

Kelly Stouffer (1987, 1st Round, 6th Overall Pick, 2nd Quarterback Drafted) - Stouffer was drafted out of Colorado State by the St. Louis Cardinals.  Notable quarterbacks taken after Stouffer in that year's draft includes Jim Harbaugh (26), Rich Gannon (98) and Steve Beuerlein (110). Stouffer did not attempt enough passes in any given year to qualify, and threw a total of 437 passes over the course of four different seasons.

Heath Shuler (1994, 1st Round, 3rd Overall Pick, 1st Quarterback Drafted) - Shuler was drafted out of Tennessee by the Washington Redskins.  He qualified in 1 year (in 1994), when he completed 45.3% of his passes, and had 4.5% of his passes intercepted, a poor enough performance to be -2.54 standard deviations below the mean.

Ryan Leaf (1998, 1st Round, 2nd Overall Pick, 2nd Quarterback Drafted) - Leaf was almost picked by the Indianapolis Colts.  Instead, they went with some fellow named Peyton Manning.  The San Diego Chargers drafted Leaf out of Washington State.  Successful quarterbacks drafted following Leaf that year are Brian Griese (91) and Matt Hasselbeck (187).  In 2000, Leaf completed 50.0% of his passes, and had a 5.6% interception rate, a performance so poor it was -2.81 standard deviations below the mean.

Tim Couch (1999, 1st Round, 1st Overall Pick, 1st Quarterback Drafted) - Couch was drafted out of Kentucky by the Cleveland Browns.  That year, the following quarterbacks were also drafted.  Donovan McNabb (2), and Daunte Culpepper (11).  Couch played enough to qualify in 3 seasons with an average standard deviation of -0.58.  His best year was in 1999, when he finished when he completed 55.9% of his passes, and had 3.3% intercepted, good for -0.42 standard deviations below the mean.

Akili Smith (1999, 1st Round, 3rd Overall Pick, 3rd Quarterback Drafted) - Smith was drafted out of Oregon by the Cincinnati Bengals.  Donovan McNabb was taken 1 pick ahead of him, and Daunte Culpepper was taken 11th that year.  Smith qualified in 1 year (in 2000) when he completed just 44.2% of his passes and 2.2% picked off, sufficient to be -2.10 standard deviations below the mean.

Alex Smith (2005, 1st Round, 1st Overall Pick, 1st Quarterback Drafted) - Smith was drafted out of Utah by the San Francisco 49ers.  Although still a little early to be evaluating this draft, a couple of quarterbacks that were taken subsequent to Smith are Aaron Rodgers (24), and Jason  Campbell (25).  Smith has qualified in 1 year (2006), when he completed 58.1% of his passes and had 3.6% of his attempts intercepted.  He finished that year -0.61 standard deviations below the mean CMI.

That's a who's who of busts and disappointments.  I could name another dozen or so who arguably could make this list, but we must draw the line somewhere!

Quarterbacks and the NFL Draft - Part 4

In my last post, we looked at quarterbacks and their expected performance as compared to where they were drafted. We concluded that their on-field performance cannot be attributed to their draft position, and perhaps those drafted earlier play a bit longer. There definitely was a correlation between draft order, and their likelihood of actually playing in the NFL (playing as defined by throwing enough passes in a given year to qualify statistically).

All we really looked at in the previous post were averages over the past 30 years. We didn't spend any time looking at individual quarterbacks. In this post, we'll do so. We'll take a stab at answering the following question: Which quarterbacks have been the surprises, or the diamonds in the rough? These would be the quarterbacks that, when drafted, either had little likelihood of actually playing in the NFL, or, even when thought that they would play, surprised us as to how well they played and/or how long they played. These would also be quarterbacks that were drafted early in the draft (perhaps even in the first round), but worked out much better than the earlier picks.

Here is my list of the 10 best surprises of the NFL Draft over the past 30 years. Admittedly, it is somewhat subjective, but I do use objective criteria to determine the list.

10. Matt Hasselbeck (1998, 6th Round, 187th Overall Pick, 6th Quarterback Drafted) - Hasselbeck was drafted by the Green Bay Packers out of Boston College behind Peyton Manning (1), Ryan Leaf (2), Charlie Batch (60), Jonathan Quinn (86), and Brian Griese (91). Hasselbeck was traded by the Packers to the Seattle Seahawks in 2001. He has thrown enough passes to have qualified in 7 seasons, and his CMI has averaged 0.14 standard deviations above the mean over that period. His best year was in 2005, where he completed 65.5% of his passes and only had 2.0% of his pass attempts intercepted, a performance good enough to be 1.50 standard deviations above the mean.

9. Brian Griese (1998, 3rd Round, 91st Overall Pick, 5th Quarterback Drafted) - Griese was drafted by the Denver Broncos out of Michigan behind Peyton Manning (1), Ryan Leaf (2), Charlie Batch (60), and Jonathan Quinn (86). Griese became the starter in 1999 following the retirement of John Elway, and in 6 qualified seasons his CMI has averaged 0.42 standard deviations above the mean. His best season was 2000, where he completed 64.3% of his passes and only had 1.2% of his passes intercepted, worthy of being 2.04 standard deviations above the mean.

8. Neil O'Donnell (1990, 3rd Round, 70th Overall Pick, 5th Quarterback Drafted) - O'Donnell was drafted by the Pittsburgh Steelers out of the University of Maryland behind Jeff George (1), Andre Ware (7), Tom Hodson (59), and Peter Tom Willis (63). He started in 1991, qualified in 5 seasons for the Steelers, and threw enough passes in 2 more seasons - one for the Jets in 1997 and one for Cincinnati in 1998. He averaged 0.47 standard deviations better than the mean over those 7 years, and his best year was 1998, when he completed 61.8% of his passes, and was intercepted 1.2% of the time, good enough to finish 1.52 standard deviations above the mean.

7. Trent Green (1993, 8th Round, 222nd Overall Pick, 8th Quarterback Drafted) - Green was drafted by the San Diego Chargers out of Indiana University behind Drew Bledsoe (1), Rick Mirer (2), Billy Joe Hobert (58), Mark Brunell (118), Gino Torretta (192), Alex Van Pelt (216), and Elvis Grbac (219). He was the last quarterback drafted in 1993. He first threw enough passes to qualify in 1998 for the Washington Redskins. He qualified in 6 seasons altogether, the last 5 with the Kansas City Chiefs from 2001-2005. Over those 6 seasons, his CMI averaged 0.26 standard deviations above the mean. His best year came in 2003, when he completed 63.1% of his passes, and had 2.3% intercepted, good enough to finish 1.18 standard deviations better than the mean.

6. Mark Brunell (1993, 5th Round, 118th Overall Pick, 4th Quarterback Drafted) - Brunell was drafted by the Green Bay Packers out of the University of Washington behind Drew Bledsoe (1), Rick Mirer (2), and Billy Joe Hobert (58). He was traded to the Jacksonville Jaguars in 1995. He qualified in 8 straight seasons there, and then 2 more in 2005 and 2006 for the Redskins. Over those 10 seasons, his CMI averaged 0.64 standard deviations above the mean. His best year was 1997, where he completed 60.7% of his passes, and had 1.6% of his passes intercepted - good enough to be 1.23 standard deviations better than the mean.

5. Rich Gannon (1987, 4th Round, 98th Overall Pick, 7th Quarterback Drafted) - Gannon was drafted by the New England Patriots from the University of Delaware behind Vinny Testaverde (1), Kelly Stouffer (6), Chris Miller (13), Jim Harbaugh (26), Cody Carlson (64), and Mark Vlasic (88). He was traded to the Minnesota Vikings and qualified in three years (1990-1992) while there. He threw enough passes to qualify again in 1998 with Kansas City, and then from 1999-2002 with the Oakland Raiders. Over the 10 qualifying seasons, he averaged 0.64 standard deviations better than the mean. His best year was in 2001, when he completed 65.8% of his passes, and only had 1.6% of his attempts intercepted. This was good enough to be 2.54 standard deviations better than the mean that year.

4. Dan Marino (1983, 1st Round, 27th Overall Pick, 6th Quarterback Drafted) - This was the "Year of the Quarterback". Marino was drafted by the Miami Dolphins out of the University of Pittsburgh behind John Elway (1), Todd Blackledge (7), Jim Kelly (14), Tony Eason (15), and Ken O'Brien (24). He started immediately, and went on to throw enough passes in 16 different seasons, averaging 0.51 standard deviations above the mean CMI. His best year was 1984, where he completed 64.2% of his passes, and had 3.0% of his passes intercepted. This was good to finish 1.28 standard deviations above the mean CMI that year. Dan Marino was elected to the Pro Football Hall of Fame in 1995.

3. Brad Johnson (1992, 9th Round, 227th Overall Pick, 13th Quarterback Drafted) - Johnson was drafted by the Minnesota Vikings out of Florida State behind David Klingler (6), Tommy Maddox (25), Matt Blundin (40), Tony Sacca (46), Craig Erickson (86), Casey Weldon (102), Will Furer (107), Chris Hakel (112), Jeff Blake (166), Kent Graham (211), Bucky Richardson (220), and Mike Pawlawski (222). He qualified in 9 different seasons - first with Minnesota (1996-1997), then the Redskins (1999-2000), then Tampa Bay (2001-2003), and finally with the Vikings again (2005-2006). Over those 9 seasons, his CMI averaged 0.73 standard deviations better than the mean. 2005 was his best year, where he completed 62.6% of his passes and only 1.4% of his attempts were intercepted, a performance good enough to be 1.32 standard deviations better than the mean CMI that year.

2. Tom Brady (2000, 6th Round, 199th Overall Pick, 7th Quarterback Drafted) - Brady was selected by the New England Patriots from the University of Michigan behind Chad Pennington (18), Giovanni Carmazzi (65), Chris Redman (75), Tee Martin (163), Marc Bulger (168), and Spergon Wynn (183). After replacing an injured Drew Bledsoe in the second game of the 2001 season, he started every game until he got injured during the first game of the 2008 season. Over the 7 year period from 2001-2007, his CMI was good enough to average 0.82 standard deviations above the mean. His best year came in 2007, when he completed 68.9% of his passes, and was intercepted 1.4% of the time, good enough for 2.14 standard deviations above the mean.  Tom Brady will likely get into the Hall of Fame.

1. Joe Montana (1979, 3rd Round, 82nd Overall Pick, 4th Quarterback Drafted) - Montana was drafted by the San Francisco Forty Niners from the University of Notre Dame behind Jack Thompson (3), Phil Simms (7), and Steve Fuller (23). Although he played every single game in the 1979 season, he only threw 23 passes that year. Beginning in 1980, he qualified every year until 1990, when injuries slowed him down. He was eventually traded to the Kansas City Chiefs, where he played for 2 more seasons. Over those 13 years, his performances on the field, he averaged 1.54 standard deviations above the mean, including the remarkable feat of never having a season below the mean. His best season was 1989, when he completed 70.2% of his passes and had 2.1% of his pass attempts intercepted, a performance so good relative to his peers, that it ranked 2.96 standard deviations above the mean.  Montana was elected to the Pro Football Hall of Fame in 2000.

Now that's a pretty good list!

First Year Quarterback Performance - An Evaluation Method

This past weekend, the New England Patriots traded Matt Cassel to the Kansas City Chiefs.  In fact, they traded linebacker Mike Vrabel and Cassel for a 2nd round pick (pick #34) in the upcoming April draft (see here for the details).  Depending on what your news source is, this was labeled as either a good deal for Kansas City, or a bad deal for New England.  The experts discussing this issue don't seem to support their arguments by any objective measures - they just opine.  I am not going to jump into the debate here.  I will offer this however - we will probably not know who will end up on the better end of this deal until many years from now, and even then, we may not know.  

What about the objective evidence?

Matt Cassel is not the only quarterback in 2008 that had a good year as a first-year quarterback.  There was Aaron Rodgers of the Packers, Shaun Hill of the 49'ers, and Matt Ryan of the Falcons.  There were four other quarterbacks who threw enough passes this year to make the list of first-year quarterbacks - Ryan Fitzpatrick of the Bengals, Joe Flacco of the Ravens, JaMarcus Russell of the Raiders and Tyler Thigpen of the Chiefs.  

Notice here that I didn't use the term rookie quarterback, but instead used the term first year.  The way I look at a quarterback, is that I only consider a quarterback who has thrown "enough" passes to qualify in a given year.  The # of passes has varied by year (as opposed to the fixed standard used by NFL.com).  For the 69-year period from 1940-2008, my database contains 1,451 passing seasons, 336 of which were by a quarterback qualifying for the first time.

How did the aforementioned 8 quarterbacks do?  

I use a statistical measure I have previously developed and discussed, called CMI (Completions Minus Interceptions, calculated as Completions/Attempts - 3*Interceptions/Attempts) instead of the NFL Passer Rating system (using the NFL passer rating system gets you essentially the same answers), and then use the standard deviation from the mean as the measuring stick (for the pool of qualifying passers, I calculate the the mean and standard deviations for each year, and relate a particular performance in that year to the mean using that year's standard deviation).  Using the mean and standard deviation this way not only allows me to compare these 8 quarterbacks relative to each other, but also relative to all quarterbacks this year, or any other year.

In a post not too long ago, I showed how, over the past 69 years from 1940-2008, when the standard deviation is used as the measuring stick, the accumulated data form a near-perfect theoretical standard normal distribution.  And I also showed how, this is true regardless of whether one used the NFL passer rating system or CMI as the statistical base.

I can now show you how these first year quarterbacks did compare to the rest of the QBs in 2008, using this measure.  But this picture may not tell us much, if anything.  Here's the graphical illustration:

Perhaps the only thing that may be apparent from this is that first year quarterbacks don't do nearly as well as quarterbacks with more experience.  Even drawing that conclusion from this is a little iffy.  However, it is clear (at least by this measure) that Cassel had the best year of the 8 first year QBs.  Rodgers was a close second.  JaMarcus Russell and Tyler Thigpen had disastrous first years.  In an earlier post regarding Jeff Garcia, I had suggested that Kansas City might be a good place for Garcia.  Scott Pioli, the new Chiefs' GM apparently was thinking the same thing - upgrade at quarterback, and he did - with Cassel.

Going back to the standard normal curve, let's look at just the first year performances.  Recall that, when all 1,451 seasons are aggregated in a histogram, it looks very much like a standard normal curve.

This doesn't quite look "normal", does it?  The first year performances appear skewed to the left, or, compared to all quarterback performances, appear to under-perform.  Since the total database is normally distributed, then performances in years 2 and beyond must obviously be skewed to the right.  This prompts the question: does experience improve performance, and, in particular, how and when?

One way to look at this is to take the entire database, and look at the average standard deviation from the mean for each additional year a player qualified.  Here's what that graph looks like.

Wow!  I didn't realize that it takes about 5 qualifying years to turn into an average quarterback.  And that's just for those playing that long!  Even though this graph goes out to 18 years, there is hardly any data beyond the 15th year (there are only 8 quarterbacks who have had qualifying seasons 16 different years).  According to this illustration, basically, first year quarterbacks, as a group, struggle mightily.  They then gradually improve each year until about year nine.  Quarterbacks, on average, must still endure sub-par years in years 2-4, and for those that survive, their reward is about four more years of improving performance, assuming they stay healthy for that long.

Note: This analysis - the method of breaking down a quarterback's expected career in terms of passing performance as it relates to experience and how it is expected to change over time, is the first I've seen published publicly.  But I digress.

Ok, so we've taken a look at these 8 QBs relative to all quarterbacks in 2008, and we've taken a look at first year quarterbacks in general.  So the next step is to take a look at the best first year seasons.  In my database, there are 336 first year seasons in all (in other words, in the 69-year period from 1940-2008, 336 different quarterbacks threw enough passes in a season to qualify at least once).  The table below shows the 75 best.

Cassel (#48) and Rodgers (#60) are the only 2 of the 8 in 2008 that crack the top 75 all-time.  On the other end of the scale, we get JaMarcus Russell (#264) and Tyler Thigpen (#290).

When Matt Ryan (#128) got off to a solid start, and the Falcons surprised many by getting into the playoffs, there were many debates as to whether his season was one of the best ever by a first year player.  Not according to this measure.  Often, during those discussions, Kurt Warner's first year was brought up.  In looking at the table above, well, he did have an outstanding first year in 1999.  It ranks #6 all-time in terms of first year seasons.  The best ever - Roger Staubach's 1971 season (Staubach was a rookie in 1966, and played in both 1969 and 1970.  However, he didn't attempt enough passes in either year to qualify).  

Notable quarterbacks high on this list: Joe Montana in 1980 (#4), Tom Brady in 2001 (#14) - (see my post earlier comparing Brady's season to Cassel's here), Brett Favre in 1992 (#15), Dan Marino in 1983 (#21), and Johnny Unitas in 1956 (#22).  

Notable absentees:  Steve Young in 1986 (#154), Peyton Manning in 1998 (#208), Joe Namath in 1970 (#202)*, and Troy Aikman in 1989 (#311).  The all-time worst first year performance - Terry Bradshaw in 1970 (#337)*.

* Keep in mind that 1970 was the first year of the NFL following the merger, and 15 quarterbacks showed up in the database as having that year as their first year, although Bradshaw's 1970 season was his actual rookie season.

Are we done?  Hardly.  So after I went through this list, and still trying to objectively value Matt Cassel, I was interested in answering the following question(s):  what does the first-year tell us about a quarterback's future potential?  Do quarterbacks who have good first years, have good careers?  What about quarterbacks who have sub-par first years?  In other words, is there a correlation between year 1 performance and subsequent years?  When in doubt, it's always a good idea to take a look at the data.

Whoa!  The correlation between year 1 performance, and career performance, as measured by CMI standard deviations relative to the mean, is 0.75.  What that really means is that 57% of the variation in a quarterback's career performance can be explained by his year 1 performance.  I find that incredible.  So I looked a little deeper.  It turns out that 112 of the 336 quarterbacks only have 1 qualifying year.  And, for that group, the correlation coefficient is, well, 100%!  If I only looked at those quarterbacks whose career included at least 2 different years in which they threw enough passes to qualify, then the correlation coefficient drops to 64%.  Still quite remarkable.  

The table below breaks down the 336 first year seasons in terms of CMI standard deviations to see if we can glean any additional insight into whether a good first year translates into a good career and vice versa.


Well, the evidence is pretty clear.  Taken as two groups, players that have better than average (remember, the mean and standard deviation is relative to all quarterbacks, not just first year quarterbacks) first years tend to have longer careers and more productive careers than those that have sub-par first years.  And, breaking it down even further (in other words, just looking at the group whose first year was better than average, or the group whose first year was worse than average), you can clearly see that even within those sub-groups, the better the first year, the longer the career (although this correlation isn't nearly as strong), and the more productive the career. 

I haven't spent too much time on my blog discussing individual careers, except in the posts discussing Brett Favre and Jeff Garcia.  If I do this en masse, it would be such a powerful post - I'll basically be giving you my list of greatest passers to ever play the game, that I need to put a lot more thought into it than I have to-date.  In any case, some of the obvious names will show up on that list.  However, you'll find some exceptions as well (and I'm sure many of you will take exceptions to the list).  The point I really want to make here is that, when combining the table above, and the discussion following the list of the top 75 first years, it is quite remarkable how Troy Aikman "escaped" the trend, and turned into a great quarterback (or, more accurately, a great passer) - he truly turned out to be an exception (as for Young and Manning, even though they didn't rank well on the first year list, their first years weren't "that bad" compared to other first year seasons - as a matter of fact, Young's was "above average" compared to a typical first year).  Notice how I don't mention Bradshaw and Namath - it is because, statistically, their careers were sub-par.  When I publish my list of all-time greatest (and worst) passers, we'll get into it in more detail.

So, in trying to look at Matt Cassel's first year performance objectively, we've looked at how quarterbacks perform as their careers progress, how quarterbacks perform in their first year, who had great first years, where Cassel's season ranks on that list, and, how a first year translates into (or, is predictive of) a career.  Now perhaps, we can view Matt Cassel's 2008 year in an objective manner, and, if so inclined, you can at least evaluate the trade armed with some data.

Naturally, while I was putting this post together a question popped into my mind, and that is:  is a quarterback's second year a good indicator of their career?  What about looking at their first and second years?  Check back in a week or two to find out.

Anybody Looking for a Good QB?

Available:  Experienced quarterback with an accurate arm with ability to make immediate contributions to team; Ideally, would like to start, but not a requirement; may be perfect as back-up; durability maybe an issue.

I am referring of course to Jeff Garcia, released today by the Tampa Bay Buccaneers.  I am personally a big fan of Garcia, because I believe he is underrated, and his achievements on the field have gone relatively unnoticed.

Let's take a look.

He began his NFL career in San Francisco in 1999, replacing not one, but two back-to-back Hall of Famers, and arguably the two greatest passers of all-time, Joe Montana and Steve Young.  There was no way Jeff could live up to those lofty expectations.  However, he did perform very well, as the table and chart below shows.  While he may not have performed at Joe Montana and Steve Young levels, his years in San Francisco were very productive.  After a sub-par year in 2003, he bounced around for three years, first as a starter in Cleveland (2004), then as a back-up in Detroit (2005) and Philly (2006).  He finished the 2006 season as the starter in Philadelphia for the last 6 games after Donovan McNabb got injured.  He showed that he could still play, and Tampa Bay, in need of a quarterback following an injury to Chris Simms, picked him up.  He did well above average each of 2007 and 2008, as you can see from the table and charts (he doesn't show up in the charts in the years 2004-2006 since he didn't attempt enough passes to qualify).  

Now, Tampa Bay, deciding to turn to the youth movement, have invested in Luke McCown, and, have deemed Garcia expendable.  

Garcia has been a very accurate as a passer throughout his career.  In a previous post, I discussed interceptions at length.  His 2007 season ranks 18th all-time (see Exhibit 3 in this post) in interceptions per attempt, in terms of standard deviations from the mean (his 2008 season ranks 118th, out of the 1,451 seasons in the database since 1940).  As a career passer, he ranks 12th all-time (see Exhibit 6 in this post).  By not throwing interceptions, he will help your team win.

I believe that Garcia has a couple of good years still left in him, if not as a starter, as a back-up.  His durability has been an issue the past two seasons, and hence, might be a liability as the starter.  

So who might be worthwhile candidates?

The following charts show, for each of 2007 and 2008, how a team's passing game stacked up in terms of standard deviations from the mean.  It's no coincidence that teams that were above average won more games than they lost, and teams that were below average lost more games than they won.  The charts show how the teams did both in terms of NFL passer rating and CMI.  Note that Garcia averaged 0.53 standard deviations above average in terms of NFL passer rating for the past two seasons, and averaged 1.23 standard deviations above average in terms of CMI.

To see which teams should be looking to get Garcia, let's start at the bottom, and move our way up.

Cleveland Browns - They have two young quarterbacks, Derek Anderson and Brady Quinn, both coming off season-ending injuries, that will compete for the starting job.  Not a candidate.  Look for another miserable year in Cleveland.

St Louis Rams - Marc Bulger is the starter, and for now at least, another soon-to-be 39 year-old - Trent Green, is the back-up.  Not a candidate.  By the way, what's happened to Bulger?  Will 2009 be more like his first few years, or his last two?  I suspect the latter.  Another long year in St Louis.

Oakland Raiders - Paying a lot of money for JaMarcus Russell.  They've also got Andrew Walter, Marques Tuiasosopo, and they just picked up Bruce Gradkowski off waivers.  I haven't a clue what the Raiders are doing, and I don't think the Raiders do either.  Not a candidate.  Perhaps we'll start paying attention when Al Davis passes away.

San Francisco Forty Niners - They're not sure who their starting quarterback is.  Shaun Hill?  J.T. O'Sullivan?  Alex Smith?  The niners maybe a candidate.  Garcia played here, and is originally from close-by Gilroy.

Detroit Lions - Daunte Culpepper is the starter.  Or is it Dan Orlovsky?  Didn't seem to matter in 2008, as they went 0-16.  If I was GM, I'd replace Culpepper with Garcia, and draft a quarterback.  But that's just me.  Garcia may not want to go here, but that's a different matter altogeher.

Chicago Bears - Kyle Orton is the starter, and Rex Grossman is the back-up.  Neither of whom are very good.  I would think that Garcia would be an excellent choice to be a back-up here.

Kansas City Chiefs - Tyler Thigpen is the starter.  Previous starter Damon Huard is the back-up.  Garcia could be a potential back-up here.

Cincinnati Bengals - Ryan Fitzpatrick finished the year as the starter, having replaced the injured Carson Palmer.  Not a candidate.

Of the remaining teams, perhaps the Minnesota Vikings might be interested.  Gus Frerotte is simply not a good option.  Maybe they'll lure Brett Favre out of retirement!  The NY Jets might be an option for Garcia as well.  Most likely, Kellen Clemens will be the starter, but Garcia could be a viable back-up here.  

So these are teams that should be interested in Garcia.  We'll wait and see where he actually ends up.  

The Importance of Interceptions (or lack thereof)

I have spent quite a bit of time lately talking about interceptions.  In case you have any doubt that an interception can make a difference you got that answer on Super Bowl Sunday.  In probably one of the greatest plays in Super Bowl history (until the catch by Santonio Holmes with 35 seconds left that gave the Steelers a come-from behind victory), with Arizona on the Pittsburgh 1 yard line, first and goal, and 18 seconds left in the first half, the Steelers' James Harrison picked off Arizona's Kurt Warner and returned it all the way for a touchdown as time expired.  This was, in effect, a 13-point play, as Arizona's expected points at the Pittsburgh 1 was about 6 points.  Brian Burke, who I've highlighted before, has an excellent post on the subject on his site at advancednflstats.com.   

Take a look at the graph below.  The blue line (on the left scale) shows the league average QB passer rating (for those QBs who thew enough passes during the season to qualify) by year since 1940.  As you know, the NFL's QB passer rating formula has four components - completion percentage, yards per attempt, touchdown percentage, and interception percentage (see my previous posts on the subject here and here).  When it was designed in 1973, the formula used the 1972 season as a "base", and hence created adjustments to each component, such that the average would be a score of 1.00 for each component, resulting in a passer rating for 66.7 for a quarterback who had average statistics in each of the four categories.  The actual calculations for each of the four components in 1972 yielded the following four figures - 1.085, 0.954, 0.897, and 1.043, respectively, which in turn yielded the average quarterback passer rating of 66.3 (the figure for the qualified leaders turns out only slightly higher - 67.9).  Back then, each of the four components were essentially balanced.

Today, it's a different story.  If the system was balanced, then we would expect the interceptions component to make up about 25% of the quarterback passer rating score.  Looking at the graph again, and this time looking at the red line (on the right scale) shows how much the interceptions component influences the league's quarterback rating system.  It hasn't been below 30% since 1983, and the last time it was "around" 25% was actually 1971.  The point is that the NFL passer rating system is not what it used to be, and interceptions are the leading weight in today's NFL passer rating system.

Let's take a look at another example.  Tom Brady's record-breaking 2007 season.  That year, Brady completed 398 of 578 passes, for 4,806 yards, and threw for 50 touchdowns while only throwing 8 interceptions.  His passer rating that year - 117.2, is second only to Peyton Manning's all-time best 121.1 in 2004.  Most people will remember the 50 touchdowns that Brady threw.  Impressive as it was, it wasn't that impressive (I'll have a post later on this subject).  Most people will not remember that he only threw 8 interceptions in 578 attempts.  That ranks as 14th best all-time in terms of single-season interception percentage (see Exhibit 2), and, it accounted for 29% of his quarterback passer rating that year (his touchdowns accounted for 25%).

There are 7 exhibits on interceptions attached to this post.   Note that for all exhibits, I only considered those passers in any given year that threw enough passes to qualify - in other words, if a quarterback threw 20 passes during a season, that would not be enough to qualify him.  On the nfl.com site, the minimum qualification standard is 14 passes per game (so, for the 2008 season, the standard would be 14 * 16 = 224 passes).  The NFL uses this standard EVERY year.  

I think this universal standard is inappropriate.  Using this method for example, only three passers qualified in 1940.  The 14 passes comes from the fact that during the 50's, 60's and 70's, the average number of passes attempted by a team during a game was about 28.  Of course, we all know that this has changed over time.  The average number of passes attempted in a game by a team was 32.3 in 2008.  It has been below 30.0 once since 1979.  

The standards that I use varies by year, and reflects the average # of passes attempted during these years.  Here are the standards:

1940-1946 - 6.5

1947-1960 - 11.0

1961-1969 - 14.0

1970-1977 - 12.0

1978-1994 - 15.0

1995-2008 - 16.0

Using these standards, I get 1,451 quarterbacks in my database, with a low of 9 quarterbacks qualifying in each of 1941 and 1943, and a high of 32 quarterbacks qualifying in each of 1999 and 2005.  For 2008, I had 30 quarterbacks in my qualified database, whereas the NFL.com has 32.  So, not a big difference in recent years.  I just think that applying a universal standard across all these years is silly, especially when the game has changed so much.

In any case, now that we got the some of the technical stuff out of the way, here are the 7 exhibits:

Exhibit 1 - Chronological list of NFL leader (lowest) in interception percentage

Exhibit 2 - Best seasons in terms of interception percentage

Exhibit 3 - Best seasons in terms of standard deviations from the mean

Exhibit 4 - Worst seasons in terms of standard deviations from the mean

Exhibit 5 - Best (lowest) career interception rate (minimum of 1,000 passes attempted)

Exhibit 6 - Best career interception rate relative to league average

Exhibit 7 - Worst (highest) career interception rate relative to league average

Exhibit 1 - Chronological list:

Nothing spectacular here, it's simply each year's best.  I observe a few things: 

Clearly, the average interception rate has been decreasing every decade.  

The best quarterbacks in a given year seem to be around 1.8 standard deviations better than the average.

Slinging Sammy Baugh led the league 4 out of 6 years during the period from 1942 to 1947 - the only quarterback to have led the league on 4 separate occasions.  

6 quarterbacks - Bart Starr, Bobby Thomason, Charlie Conerly, Ken Anderson, Ken O'Brien, and Roger Staubach have led the league on 3 different occasions.

5 quarterbacks led the league in consecutive seasons - Sammy Baugh, Bobby Thomason, Milt Plum, Ken Anderson and Ken O'Brien.

Only 2 quarterbacks in history, Steve DeBerg in 1990 (0.90%), and David Garrard in 2007 (0.92%) completed a season where less than 1 percentage of their attempted passes were intercepted (this is not entirely evident by looking at Exhibit 1, but can be confirmed by Exhibit 2).  What's most unusual about DeBerg's performance that year was the fact that of the four interceptions he threw during the season, three of them were in one game!  In other words, he threw 1 INT the rest of the season.  Let's take a look at Exhibit 2.

Exhibit 2 - Best seasons in terms of interception percentage:

So indeed, only 2 quarterbacks have had seasons with fewer than 1% interceptions.  The list below is the kind of list that would show up in a record book.  

You'll see, for example, that Jason Campbell's 2008 season ranks in the top 5 all-time, and 2 other quarterbacks in 2008, Chad Pennington and Jeff Garcia, also had noteworthy seasons, with both finishing in the Top 30 all-time.  

If you look carefully, you'll notice that the list is dominated by quarterbacks in the past 2 decades.  Of the Top 50, 45 have occurred since 1990.  21 last decade and 24 this decade.  The 5 seasons in the top 50 not to have occurred in the last 19 years are Steve Bartkowski's 1983 season (#3), Bart Starr's 1966 and 1964 seasons, respectively (#7 and #18), and Ken O'Brien's 1985 and 1988 season's respectively (#36 and #38).  

So, what are we saying?  Are we saying that quarterbacks prior to 1990 were not very good?  No, not at all.  The league has changed.  From Exhibit 1, you can see that the average interception rate has been decreasing every decade.  So, we simply can't just compare a quarterback from one decade to another.  That's where Exhibit 3 comes in.  Let's take a look at that.

Exhibit 3 - Best seasons - Interceptions percentage, ranked by how different the particular season was compared to the mean, using the standard deviation as the measuring stick:

Wow, what a difference.  You can quickly see that this is a much better representation of the past 7 decades.

1940's - 3

1950's - 2

1960's - 10

1970's - 9

1980's - 7

1990's - 6

2000's - 13

Look at Bart Starr!  Ranked twice in the Top 5, and 3 times in the Top 25.  

And, this measure doesn't discount Steve DeBerg's and David Garrard's great seasons - they're both still in the Top 5.  But it does give one a bit more perspective.  In other words, this suggests that Bart Starr's 1962 season (which ranks #430 in absolute terms), when compared to everyone else's performance during that season, was better than David Garrard's performance, when compared to how all the other quarterbacks did in 2007.

Now for a little math.  Why are we using standard deviation as a measure of separation?  And why does using it make comparing quarterbacks across years more meaningful?  

A non-technical definition of the standard deviation is that it is a measure of the dispersion of a set of data around the average.  By dispersion we mean spread.  Knowing the average of the data, and knowing how spread the data is, we can try to determine how likely a given observed value is.  We can use this data to compare different data sets, and relate them to one another.   So for example, in a data set where the average is 5, and the standard deviation is 2, an observation of 8, would mean 1.5 standard deviations ((8-5)/2 = 1.5) above the the mean.  In a data set where the average was 7, and the standard deviation was 4, a value of 13 would also be 1.5 standard deviations from the mean.   You can now see why the use of a standard deviation could be useful in comparing different sets of data.  

As you can see from Exhibit 1, the league average interception rate has been changing over time.  In addition, while I have not shown it explicitly, the standard deviation around the mean has also been changing.  As a matter of fact, in the early 1940's, the standard deviations were quite high because there were fewer players involved, the number of attempted passes were fewer, and arguably the talent pool was not as great (i.e. passing was a fairly new concept).  

By relating a given observation of an interception rate in 1943 to the mean interception rate that year, and the standard deviation of interception rate that year, one can then compare that particular observation to an observation of an interception rate in 2003, by its relationship to the mean and standard deviation of interception rates in 2003.  If one assumes, in particular,  that in any given year, that all observations about the mean are distributed normally (i.e. a "bell-shaped" curve), then the comparisons become that much more meaningful.  For example we know that in a standard bell-curve, that approximately 68% of the observations will fall into a band +/- 1 standard deviation from the mean, and approximately 95% of observations will fall into a band +/- 2 standard deviations from the mean.  Also, if, after relating the observations in each year to each year's mean and standard deviation, one aggregates the data across all years (since the data has been "normalized" to the same scale - a number in relation to a mean and standard deviation; in the example earlier, the observation of 8 in the first data set has the same value on a normalized basis as the observation of 13 in the second data set - 1.5), then the aggregated data should look like a standard normal curve, with a mean of 0, and a standard deviation of 1.  

Well, I went through the trouble of doing that, and guess what - that is exactly what the looks like.  All 1,451 qualified passers over the 69 years from 1940 to 2008 were analyzed in this manner, and the result is in the graphical illustration below.  The average for the entire data set is -0.06 (close to 0), and the standard deviation is 1.00!  Also, it turns out that 69% of the observations are within +/- 1 standard deviation, and 96% are within +/- 2 standard deviations.  Quite a remarkable achievement!  I've graphed a standard normal distribution as well, so that you can see for yourself how close the actual data is to the theoretical curve.

Exhibit 4 - Same as Exhibit 3, except ranks the worst seasons:

Terry Bradshaw's 1970 rookie season ranks as the all-time worst.  And Vinny Testaverde's 1998 campaign is not that far behind.  Although Testaverde's rookie season was 1987, he didn't "get exposed" until 1988.  Most recently, Gus Frerotte's 2008 campaign was an absolute disaster.  There's some names on the list that I would not have expected to see (let's face it, these are the 50 worst performances over the past 69 years - 1,451 quarterback seasons) - names like Favre and Aikman, along with the aforementioned Bradshaw.  What was even more surprising to me was that they each showed up not once, but twice!

So now you've seen the best and the worst seasons.  How about careers?  Let's take a look at Exhibit 5.

Exhibit 5 - Lowest career interception rate - absolute figures - with a minimum of 1,000 passes attempted:

For reasons I discussed above, I don't like this list that much, as it is biased towards the more recent years.  But, since people like looking at these types of lists, I have included it.  Exhibits 6 (best) and 7 (worst) reorder the data after a player's career has been compared to the average during their career.

Exhibits 6 (best/lowest) and 7 (worst/highest) - career interception rate relative to league average during the same time:

First question is: why, after the long dissertation about standard deviations, am I not using that as the measuring stick?  Simple answer - it's difficult!  It was a relatively simply exercise to calculate standard deviations for each year.  And it's not that difficult to do it for a given set of consecutive years.  Where it becomes increasingly difficult is to do it for every combination of multiple years, over a 69-year period.  At some point before the 2009 NFL season, I'll have it done.  That will be a better reflection of who had good or bad careers, but, in the meantime, this will have to suffice.  It's the next best thing.  It is most certainly better than the absolute comparison shown in Exhibit 5.

Ok, on to a few observations.

Exhibit 6 is a who's who of quarterbacks in football history.  Taking out the 13 players who are either currently playing, or who have retired in the past five years (i.e. not Hall of Fame eligible), 14 of the remaining 37 are in the Hall of Fame.  Let's look at it another way.  There are 27 quarterbacks who are in the Hall of Fame, who played football post 1940.  And two of them, Arnie Herber, and Clarence (Ace) Parker, played much of their careers before 1940.  Of the remaining 25, 14 show up in Exhibit 6.  

Who are the 14?

Joe Montana (#3)

Roger Staubach (#9)

Steve Young (#14)

Sammy Baugh (#19)

Fran Tarkenton (#20)

Bart Starr (#22)

Sonny Jurgensen (#23)

Otto Graham (#32)

Dan Marino (#33)

Sid Luckman (#37)

John Elway (#39)

Len Dawson (#42)

Johnny Unitas (#46)

Norm Van Brocklin (#50)

And 2 more, Y.A. Tittle (#53), and Troy Aikman (#55) just missed being in the Top 50.  Not a bad list.

What about Exhibit 7?  3 Hall of Famers, Joe Namath, George Blanda, and Terry Bradshaw are in the worst 50 all-time, in terms of their career interception percentage as it relates to the league average during the time that they played.