Pro Football's All-Time Best Passers

Now that I've got a method for evaluating a quarterback as a passer and can use it compare his career to that of others, I thought it might be illustrative to compare the Top 50 on my list to how they rank on the all-time passer rating list provided by the Pro Football Hall of Fame.  The list provided by the Hall only includes those passers who've attempted 1,500 or more passes in their career.  Also, as of today (April 27, 2010), the Hall of Fame list does not include stats from the 2009 season.  As I've mentioned in the past, the HOF list unfairly penalizes those players who played in an era where passing was secondary.  Why?  Because of the restriction of limiting it to only those passers attempting 1,500 or more passes.  So, to supplement the list from the HOF, I'll show you their passer rating ranking based on my list of 416 passers, which is a broader and more inclusive list, especially of those who played in the 1930's - 1970's.  My list includes any quarterback who qualified in at least one season.

Here's the top 50 list of All-Time Best Passers:

My list of 50 not only covers the entire spectrum of pro football history - from the beginning in the 1930's to current players, but it also presents the players who played in earlier years fairly, and in context.  The question isn't whether Joe Montana was a better passer than Sammy Baugh, but, whether Joe Montana relative to Montana's peers, was a better passer than Sammy Baugh was relative to Baugh's peers.  The HOF list can be used to answer that question, since a measure relative to a fixed standard (in other words the passer rating system) can be used to do that.  However, the HOF list cannot be used to answer the next question - how does one compare Sammy Baugh to Joe Montana given that they both played an entirely different game, with different rules and circumstances.  A fixed standard cannot solve that particular puzzle.  The use of a mean and standard deviation by year (z scores) which in turn we convert to percentile ranks can be used.  C7 is simply a quarterback's 7 best years as a passer in terms of percentile rank summed up.

3 Things I Would Change About The NFL If I was NFL Commissioner

This past season, we saw a great football game between the New Orleans Saints and the Minnesota Vikings in the NFC Championship game.  In overtime, the Saints defeated the Vikings, and, as a result, headed to Superbowl XLIV, where they eventually defeated the Indianapolis Colts.  It was a story-book ending to their season.


For Minnesota Vikings fans it was a bitter conclusion to what they were hoping was also going to be a story-book ending.  There they were, driving down the field in the waning moments of the fourth quarter when Brett Favre threw a fateful interception.  The Vikings never touched the ball after that.  At the start of the overtime, the Vikings lost the coin toss, and the Saints took the ball, drove down the field, and little known Garrett Hartley kicked the winning field goal.


This made me wonder about the merits of the overtime rules as they are, as well as other musings I've had on the game.


In this post, I will discuss 3 things that I would change about the game that would (in my opinion) make the game better.  Different, but better.

1. Points After Touchdown
2. Field Goals
3. Overtime

Point After Touchdown


The point after attempt in the NFL is worth one point, if the team chooses to kick the ball, and the ball goes between the uprights.  The kick is placed on the 2-yard line, in effect making it a 19-yard field goal.  The team does have an option of either passing the ball into the end zone or running it in, and, if successful, the point after attempt is then worth two points.  Here are the success rates of the one point point-after attempts in the NFL in each of the last three seasons:


2007  -  1,165/1,177 = 98.98%
2008  -  1,170/1,176 = 99.49%
2009  -  1,165/1,185 = 98.31%


2007 - 2009  -  3,500/3,538 = 98.93%


But that's not all.  I mentioned earlier that a point after attempt (PAT) is the same as kicking a 19-yard field goal.  Over the past three seasons, the success rate on field goals 0-19 yards are 100% (35/35).  As a matter of fact, the last kicker to miss a field goal less than 20 yards was Kris Brown, then with the Houston Texans, all the way back in 2002.


My Proposed Change(s)


Eliminate the point after kick, as well as any field goal attempt less than 20 yards.


I offer two alternatives in lieu of this:


First, after each touchdown, you could place the ball on the 2-yard line, or maybe even the 3-yard line, and you must either run it in, or throw it into the end zone.  A successful attempt would be worth 2 points.  I have read that the success today on a two-point conversion is somewhere between 50%-55%.  If the ball was placed on the 3-yard line, presumably this would decrease to about 45%.


Second, if you choose, you could place the ball on the 17-yard line, and kick the point-after attempt.  This would be equivalent to kicking a 34-yard field goal.  A successful conversion would be worth 1 point.  Here are the success rates of field goals 30-39 yards for each of the past three seasons.


2007  -  253/279 = 90.68%
2008  -  286/321 = 89.10%
2009  -  240/287 = 83.62%


2007 - 2009  -  779/887 = 87.82%


Both of these scenarios would have about the same expected point value - about 0.9 points.  


One of the biggest reasons sports is exciting is the uncertainty of the outcome.  Moving away from a basically meaningless play (the current PAT) and converting it into both a strategic decision and introducing additional elements of uncertainty would create tremendous excitement.  In my opinion, it would also reduce the likelihood of there being a tie game at the end of regulation.


Field Goals


Two rules that changed the game of basketball for the better and made it more exciting for the fans are the introduction of the shot clock in 1954, and the introduction of the 3-point shot in the 1979-1980 season.  It is hard to argue that these two elements make the game less exciting today than it was before these inventions.  A given team always has the chance of "going for the three".  The success rate is lower, but the reward is better.  The basic premise of the 3-point shot is that the further out you are from the basket, the tougher the shot, and hence the higher the point value.  There is a single line, basically 23 feet 9 inches from the basket (the reason it's not exactly 23 feet 9 inches, and other facts about the three point shot are discussed here).


The following table shows field goal accuracy in the NFL, by distance, over each of the past three seasons.

Makes perfect sense, doesn't it?  The further out you are, the more difficult the kick.  Also, the further out you are in attempting a kick, the more likely it is to get blocked.


My Proposed Change(s)


Vary the value of the field goal depending on the distance.  Remember I said earlier that no field goal less than 20 yards should be allowed.  I also placed the point-after attempt on the 17-yard line, and gave it 1 point.  Here's how I would break down the values assigned to field goals.


0-19 yards = Cannot Attempt; must go for the touchdown
20-39 yards = 1 point
40-49 yards = 2 points
50 or more yards = 3 points


I know what many of you are probably thinking.  This would decrease the number of points scored in a game.  I disagree.  The notion that this change would decrease the number of points scored by a team in a game is based on the presumption that the coaches will call the plays the same way as they do today.  They won't.  I think that two things will happen.


First, inside the 2-yard line, since the team has to go for it, the expected outcome should be greater than 3 points (6 points * 55% plus either 2 points * 55% or 1 point * 90% - see PAT discussion above).  In addition, if the team fails to make it, the other team will be in a fairly bad spot, and there's a significant chance that the team that failed to make the touchdown gets the ball back with great field position.


Second, when the team is further out, say on the 10-yard line, I think that more teams will "go for it" on fourth down when they're faced with the possibility of only getting 1 point if they kick the field goal.  In other words, the lure of the extra 6 (or 7) points would be worth the risk.  Today, the lure of an extra 3 points (versus a 3-point field goal) isn't worth the risk in coaches minds.  David Romer, in an excellent paper, suggested that coaches today are making a big mistake by not going for it on fourth down more often than they do (he was referring to the general scenario, and not specifically being inside the opponent's 17 yard line).  If Romer is right (and I think that he is), then the new scenario ought to give the coaches that extra incentive.  The fans would prefer it, and it would make the game more exciting.


Overtime


There have been numerous blog posts since the Vikings-Saints game discussing the overtime rules, and suggestions on how to change the rules such that the outcomes are "fairer".  Even the NFL is contemplating changing the current format.  The basic premise being that the winner of the coin toss (who invariably elects to receive the ball) wins approximately 60% of the time, and 2/3 times in that scenario, the loser of the coin toss doesn't even get a chance to touch the ball on offense.


Brian Burke at Advanced NFL Stats in particular has done an analysis that suggest that today's overtime rules are tremendously unfair to the loser of the coin toss.  Instead of rehashing his analysis (you can read it here), I'll merely point out that I agree with his assessment.  In his article, he (and commenters) offers some suggestions.  Although these suggested improvements are quite valid, I believe that they are mostly theoretical and appeal only to those who have an intellectual bent.  These alternatives are not practical, and therefore, I believe, unlikely to be implemented by the NFL.


The NFL's proposed changes make it slightly better for the loser of the coin toss, but it still isn't a 50/50 proposition.  As such, I do not agree with the proposed changes by the NFL.


My Proposed Change(s)


First, no overtime games during the regular season.  What's wrong with a tie?  Especially, if the current overtime rules dictate that the way in which the winner is determined is basically unfair.  Might as well flip a coin (actually, given today's rules, it would be better to flip a coin).


Second, winner in overtime must score at least 7 points in the overtime, win by at least 4 points, and, if the team receiving the ball first scores on its opening possession, then the other team must be given a chance to respond.


Many have suggested that receiving the ball second is advantageous since they know what they need to do win the game, but since the requirements are that a team scores at least 7 and wins by at least 4, the second team has to score at least a touchdown to win on its first possession.  The win by 4 requirement makes it fairer than it is today for both teams.  If the team that receives possession first scores a field goal, it cannot win by that score alone.  The proposed change is a little bit like the tie-breaker rule in tennis, where one player gets one serve to start the tie-breaker, but then each player alternates two serves at a time, with the requirement that the winner score at least 7 points, and win by at least 2 points.


Conclusions


If all three of my proposed changes are adopted, the game would be a lot different than the game played today.  There wouldn't be any meaningless extra points kicked after a touchdown, teams would "go for it" more often than they do today, points scored would be quite different, as each decision becomes strategic, more points would be scored, there would be less likelihood of a tie at the end of regulation, in the event of a tie, there would be no overtime games during the regular season, and the overtime rules in playoffs would be fairer to the team that loses the coin toss.

2009 NFL Season - Passer Ratings

We now have data for the entire 2009 season, and I can calculate the best and worst performers for the season, using my standard measure, CMI - or more appropriately, Standard Deviations from mean CMI.  CMI is easily calculated, as it is simply [pass completion percentage] - 3 * [interception percentage].  I can calculate that at any point during the season.  In order to calculate the standard deviations from the mean CMI, I like to wait until the season is over (actually, even this can be calculated at any point during the season, but it's a lot simpler for me to just do it at the end of the season - perhaps next year, I can do this after every week during the regular NFL season).   For those of you familiar with statistics, the measure of standard deviation from the mean is also called a z-score (or z-value).  And, furthermore, z-score can be very easily converted to a percentile rank (assuming that the population is normally distributed.  In a later post, I will show that z scores for CMTI or NFL passer rating over the years is distributed normally).  The beautiful thing about a percentile rank is that it is always a number between 0 and 100.  What's not to like about that!

A quick couple of notes here regarding CMI, and many of the calculations that I will be using from here on out. While I will go through the exercise of creating arduous, complex and sometimes arbitrary (and even unnecessary) calculations, I will strive for simplicity whenever possible when I present my ideas, as I believe they are more likely to be accepted.   The use of percentile ranks is an example of this.

Last year, when I first developed CMI, and especially as I looked back in time, I had to figure out how many players to use each year to determine the number of qualifiers.  I went through some elaborate mechanisms to calculate the number of passes attempted during the season (per team per game), and some qualifying standard.  As you are aware, the NFL uses 14 passes per game as the standard.  And yes, while it is simple to use, I am not a big fan of that standard since the average # of passes attempted per game has changed (increased) over time, as you can see from the table below.

The standard today should be closer to 16.

Last year, I arbitrarily assigned a factor close to 50% (with adjustments for the earlier years) to the total # of passes attempted by each team per game to determine the # of qualifiers.  I realized this year, that if I simply used the # of teams in the league as the cutoff for determining the # of qualifiers, I get fairly close to my original method.  And because it is simple, easy to explain, does a better job (in my opinion, anyway) than the current NFL standard, I will use that as the cutoff.  The table below shows the # of qualifiers by season using the NFL standard, my original standard, and my current standard.

As you can see, the # of qualifiers, regardless of which method you select is fairly close in each year, with the exceptions being the earliest years, where the NFL method simply excludes too many passers (in my opinion anyway).  Also note that the the AAFC is excluded by the NFL altogether, while the AFL and NFL have been combined in the years 1960-1969 by the NFL.  This doesn't make sense to me, since each league played a different # of games (and had a different # of teams).  Regardless, if you combine the AFL and NFL results from my method, you get close to the #s used by the NFL.  While I think that my method works, I also recognize that this standard may not be the most appropriate for future NFL seasons.

OK, back to the passer ratings.

Because I will discuss other passer rating systems/calculations that are related to CMI in later posts, I will take this moment to re-classify CMI as CMTI (acronym stands for Completions Minus Three times Interceptions).  The definition itself hasn't changed - just the acronym.

The table below shows each of the qualifying quarterbacks with their 2009 season's statistics, their NFL passer rating, the percentile rank of their 2009 NFL passer rating (in other words, the converted z-scores), their 2009 rank in terms of NFL passer rating, their 2008 NFL passer rating, the percentile rank of their 2008 NFL passer rating, their 2008 NFL passer rating rank, their 2009 CMTI, 2009 CMTI percentile rank, their 2008 CMTI and 2008 CMTI percentile rank, and their 2008 CMTI ranking, and finally, their average 2-year CMTI percentile rank (again a # between 0 and 100), together with that ranking.

I don't think there's any question as to who the elite quarterbacks are.  Also notice the struggles of first-year quarterbacks in each year.

Happy New Year!

Hi Everyone.

Happy New Year to you all.

It's been awhile since I've posted.  The reasons are many, so I shall not go into them.  Regardless, I am back posting.

I have been able to do some additional research in the meantime.  I have added data from the 1932 - 1939 NFL seasons to my database.  I have also added the 1946-1949 AAFC seasons to the database as well as the 1960-1969 AFL seasons.  And of course, updated for the just concluded 2009 NFL season.

I think that adding the data from the AAFC in the late 1940's as well as the AFL during the 1960's makes for a much richer database.  There are those however, who do not believe that including that data makes sense.  I disagree.

Having spent some time thinking about my pursuit here (creating a new passer rating system for the NFL), I've created new statistical measures in addition to those I created last year that I will discuss in upcoming posts.  I've also improved upon my original concept of CMI.

Anyhow, I'm glad to be back posting and look forward to a terrific 2010.

Cheers,
Kiran

Was Steve McNair A Good Passer?

Earlier this month, Steve McNair was shot and killed. It was a story that caught everyone by surprise, and there was much said about his contributions to the game, and how he played the game. I thought it might be worthwhile to digress briefly from my current topic - the economics of the NFL Draft to take a look back at McNair's career.

Steve McNair was drafted in 1995 out of little-known Alcorn State. The 3rd overall pick in the draft, he was the 1st quarterback taken that year. The then Houston Oilers (now Tennessee Titans) drafted him ahead of current Titans quarterback Kerry Collins.

He started out his career backing up Chris Chandler in both 1995 and 1996 while the team was still the Houston Oilers. The first year in Tennessee, he took over the starting job. He had a sub-par first year (as most quarterbacks tend to do), and then went on to have a fine career. He is of course, most notably remembered for coming one yard short of leading the Titans to a victory against the St Louis Rams in Superbowl XXXIV. The table and chart below illustrate his career in terms of the standard deviations from the mean CMI.

As you can see, from 1998 until his retirement after the 2007 season, he finished each year at above the mean CMI, with the exceptions of the 2004 and 2007 seasons which were cut short due to injury. His best year was 2003, where he finished 1.40 standard deviations above the mean CMI. His statistics that year: 62.5% completion rate, and a 1.75% interception rate. For his performance on the field that year, he was named Co-NFL MVP along with Peyton Manning. He was traded to the Baltimore Ravens following the 2005 season.

All told, Steve McNair had 9 qualifying years in his 13-year career. Of the 51 quarterbacks since 1940 to throw enough passes to qualify in 9 or more seasons, he ranks 17th. His career average standard deviation from the mean CMI during his 9 years was 0.46, putting him in the company of quarterbacks such as Norm Van Brocklin, John Brodie and Sid Luckman. The chart below illustrates the 51 quarterbacks, and Steve McNair's standing among them.

NFl Draft Economics - Part 1 - Introduction

Here I will begin a series of posts discussing the economics of the NFL Draft.  The NFL Draft puzzles me.  Every year.  There are many economists (those who are both smarter and more qualified than I am) who've tried to explain it, and I'll cover their analyses in the posts to come. 

Naturally, this is a rather difficult topic to address in one simple post, so I'll do it over a few.  It's a very broad topic - it covers the evolution of the game, identifying and measuring talent, economics, labor issues, psychology, and human behavior, among others.  I won't go into all these topics, but will address some of them either directly or indirectly.  

Through all of this, the fundamental question I'll be trying to answer is why did the Detroit Lions take Matthew Stafford as the 1st pick in 2009 Draft, and pay him $78 million (actually, he has the potential to earn as much as $78 million over six years, and about $42 million is "guaranteed" - the reason the "guarantee" is in quotes is that while that is the quoted number in the press, his actual true guaranteed money is about $17 million).

And why did they do this after they took Joey Harrington with the 3rd pick overall in 2002?  And why did they do that after they took Andre Ware with the 7th overall pick in 1990?  And why did they do that after they took Chuck Long with the 12th overall pick in 1986?  That is the set of questions I'll attempt to answer.  There must be a reason why.  All the evidence suggests that they shouldn't have done this.  So why did they do this?

Quarterbacks and the NFL Draft - Part 5

This post is the corollary to the previous post.  That is, we will attempt to answer the following question: which quarterbacks drafted in the past 30 years turned out to be the biggests busts?  Naturally, these would be the quarterbacks drafted with really high expectations - those that were drafted in the first round, or better yet, drafted in the Top 10, or even better, #1 overall, perhaps.  These cases of buyers remorse are further exacerbated when quarterbacks taken later (much later in some cases) turn out to have far superior careers.  

The list you see below, along with the list in my previous post, should illustrate the difficulty of identifying who might be the best NFL quarterbacks before their first game.  It is certainly an inexact science.  It is why I question the merits of a) picking a quarterback so early in the draft, and b) paying them such large sums of money before they ever throw a pass.  I will explore these questions in my next post.  

In the meantime, here's my list of the 10 quarterbacks who were the biggest busts of the NFL Draft over the past 30 years.  Unlike the biggest surprises, this list is a bit more difficult to assess.  For the surprises, one merely looks at the relevant statistics, and then finds an appropriate weight (for example, how much weight to assign to longevity).  For the busts, how does one quantify those that don't play (or don't play enough).  In my system of quantifying quarterback passing statistics, one must throw enough passes in a given season to qualify to be ranked (in other words, I don't rank every player that threw a pass as that would be meaningless).  For that reason, I shall merely list the players, and not rank them, starting with the earliest drafts.

The List

Jack Thompson (1979, 1st Round, 3rd Overall Pick, 1st Quarterback Drafted) - Thompson was drafted out of Washington State by the Cincinnati Bengals.  He was taken ahead of Phil Simms (7), and Joe Montana (82).  He qualified in 1 year.  He makes the list not because of his statistics as much as he was drafted so far in advance of the the quarterback who arguably was the greatest passer of all time.

Rich Campbell (1981, 1st Round, 6th Overall Pick, 1st Quarterback Drafted) - Campbell was drafted out of The University of California by the Green Bay Packers.  He was taken ahead of Neil Lomax (33).  Campbell didn't attempt enough passes in any year to qualify, and threw a total of 68 passes in two different seasons.

Art Schlichter (1982, 1st Round, 4th Overall Pick, 1st Quarterback Drafted) - Schlichter was drafted out of Ohio State by the Baltimore Colts.  The Chicago Bears selected Jim McMahon (5)  out of Brigham Young one pick later.  What bust list would be complete without Schlichter, known more famously for his gambling addictions.

Todd Blackledge (1983, 1st Round, 7th Overall Pick, 2nd Quarterback Drafted) - Blackledge was drafted out of Penn State by the Kansas City Chiefs.  Quarterbacks taken after Blackledge that year include Jim Kelly (14), Tony Eason (15), Ken O'Brien (24) and Dan Marino (27).  Todd qualified in 1984, a year when he completed 50.0% of his passes and had 3.7% of his attempts picked off.  John Elway was drafted 1st Overall that year.  Elway, Kelly and Marino are all in the Pro Football Hall of Fame.

Kelly Stouffer (1987, 1st Round, 6th Overall Pick, 2nd Quarterback Drafted) - Stouffer was drafted out of Colorado State by the St. Louis Cardinals.  Notable quarterbacks taken after Stouffer in that year's draft includes Jim Harbaugh (26), Rich Gannon (98) and Steve Beuerlein (110). Stouffer did not attempt enough passes in any given year to qualify, and threw a total of 437 passes over the course of four different seasons.

Heath Shuler (1994, 1st Round, 3rd Overall Pick, 1st Quarterback Drafted) - Shuler was drafted out of Tennessee by the Washington Redskins.  He qualified in 1 year (in 1994), when he completed 45.3% of his passes, and had 4.5% of his passes intercepted, a poor enough performance to be -2.54 standard deviations below the mean.

Ryan Leaf (1998, 1st Round, 2nd Overall Pick, 2nd Quarterback Drafted) - Leaf was almost picked by the Indianapolis Colts.  Instead, they went with some fellow named Peyton Manning.  The San Diego Chargers drafted Leaf out of Washington State.  Successful quarterbacks drafted following Leaf that year are Brian Griese (91) and Matt Hasselbeck (187).  In 2000, Leaf completed 50.0% of his passes, and had a 5.6% interception rate, a performance so poor it was -2.81 standard deviations below the mean.

Tim Couch (1999, 1st Round, 1st Overall Pick, 1st Quarterback Drafted) - Couch was drafted out of Kentucky by the Cleveland Browns.  That year, the following quarterbacks were also drafted.  Donovan McNabb (2), and Daunte Culpepper (11).  Couch played enough to qualify in 3 seasons with an average standard deviation of -0.58.  His best year was in 1999, when he finished when he completed 55.9% of his passes, and had 3.3% intercepted, good for -0.42 standard deviations below the mean.

Akili Smith (1999, 1st Round, 3rd Overall Pick, 3rd Quarterback Drafted) - Smith was drafted out of Oregon by the Cincinnati Bengals.  Donovan McNabb was taken 1 pick ahead of him, and Daunte Culpepper was taken 11th that year.  Smith qualified in 1 year (in 2000) when he completed just 44.2% of his passes and 2.2% picked off, sufficient to be -2.10 standard deviations below the mean.

Alex Smith (2005, 1st Round, 1st Overall Pick, 1st Quarterback Drafted) - Smith was drafted out of Utah by the San Francisco 49ers.  Although still a little early to be evaluating this draft, a couple of quarterbacks that were taken subsequent to Smith are Aaron Rodgers (24), and Jason  Campbell (25).  Smith has qualified in 1 year (2006), when he completed 58.1% of his passes and had 3.6% of his attempts intercepted.  He finished that year -0.61 standard deviations below the mean CMI.

That's a who's who of busts and disappointments.  I could name another dozen or so who arguably could make this list, but we must draw the line somewhere!

Quarterbacks and the NFL Draft - Part 4

In my last post, we looked at quarterbacks and their expected performance as compared to where they were drafted. We concluded that their on-field performance cannot be attributed to their draft position, and perhaps those drafted earlier play a bit longer. There definitely was a correlation between draft order, and their likelihood of actually playing in the NFL (playing as defined by throwing enough passes in a given year to qualify statistically).

All we really looked at in the previous post were averages over the past 30 years. We didn't spend any time looking at individual quarterbacks. In this post, we'll do so. We'll take a stab at answering the following question: Which quarterbacks have been the surprises, or the diamonds in the rough? These would be the quarterbacks that, when drafted, either had little likelihood of actually playing in the NFL, or, even when thought that they would play, surprised us as to how well they played and/or how long they played. These would also be quarterbacks that were drafted early in the draft (perhaps even in the first round), but worked out much better than the earlier picks.

Here is my list of the 10 best surprises of the NFL Draft over the past 30 years. Admittedly, it is somewhat subjective, but I do use objective criteria to determine the list.

10. Matt Hasselbeck (1998, 6th Round, 187th Overall Pick, 6th Quarterback Drafted) - Hasselbeck was drafted by the Green Bay Packers out of Boston College behind Peyton Manning (1), Ryan Leaf (2), Charlie Batch (60), Jonathan Quinn (86), and Brian Griese (91). Hasselbeck was traded by the Packers to the Seattle Seahawks in 2001. He has thrown enough passes to have qualified in 7 seasons, and his CMI has averaged 0.14 standard deviations above the mean over that period. His best year was in 2005, where he completed 65.5% of his passes and only had 2.0% of his pass attempts intercepted, a performance good enough to be 1.50 standard deviations above the mean.

9. Brian Griese (1998, 3rd Round, 91st Overall Pick, 5th Quarterback Drafted) - Griese was drafted by the Denver Broncos out of Michigan behind Peyton Manning (1), Ryan Leaf (2), Charlie Batch (60), and Jonathan Quinn (86). Griese became the starter in 1999 following the retirement of John Elway, and in 6 qualified seasons his CMI has averaged 0.42 standard deviations above the mean. His best season was 2000, where he completed 64.3% of his passes and only had 1.2% of his passes intercepted, worthy of being 2.04 standard deviations above the mean.

8. Neil O'Donnell (1990, 3rd Round, 70th Overall Pick, 5th Quarterback Drafted) - O'Donnell was drafted by the Pittsburgh Steelers out of the University of Maryland behind Jeff George (1), Andre Ware (7), Tom Hodson (59), and Peter Tom Willis (63). He started in 1991, qualified in 5 seasons for the Steelers, and threw enough passes in 2 more seasons - one for the Jets in 1997 and one for Cincinnati in 1998. He averaged 0.47 standard deviations better than the mean over those 7 years, and his best year was 1998, when he completed 61.8% of his passes, and was intercepted 1.2% of the time, good enough to finish 1.52 standard deviations above the mean.

7. Trent Green (1993, 8th Round, 222nd Overall Pick, 8th Quarterback Drafted) - Green was drafted by the San Diego Chargers out of Indiana University behind Drew Bledsoe (1), Rick Mirer (2), Billy Joe Hobert (58), Mark Brunell (118), Gino Torretta (192), Alex Van Pelt (216), and Elvis Grbac (219). He was the last quarterback drafted in 1993. He first threw enough passes to qualify in 1998 for the Washington Redskins. He qualified in 6 seasons altogether, the last 5 with the Kansas City Chiefs from 2001-2005. Over those 6 seasons, his CMI averaged 0.26 standard deviations above the mean. His best year came in 2003, when he completed 63.1% of his passes, and had 2.3% intercepted, good enough to finish 1.18 standard deviations better than the mean.

6. Mark Brunell (1993, 5th Round, 118th Overall Pick, 4th Quarterback Drafted) - Brunell was drafted by the Green Bay Packers out of the University of Washington behind Drew Bledsoe (1), Rick Mirer (2), and Billy Joe Hobert (58). He was traded to the Jacksonville Jaguars in 1995. He qualified in 8 straight seasons there, and then 2 more in 2005 and 2006 for the Redskins. Over those 10 seasons, his CMI averaged 0.64 standard deviations above the mean. His best year was 1997, where he completed 60.7% of his passes, and had 1.6% of his passes intercepted - good enough to be 1.23 standard deviations better than the mean.

5. Rich Gannon (1987, 4th Round, 98th Overall Pick, 7th Quarterback Drafted) - Gannon was drafted by the New England Patriots from the University of Delaware behind Vinny Testaverde (1), Kelly Stouffer (6), Chris Miller (13), Jim Harbaugh (26), Cody Carlson (64), and Mark Vlasic (88). He was traded to the Minnesota Vikings and qualified in three years (1990-1992) while there. He threw enough passes to qualify again in 1998 with Kansas City, and then from 1999-2002 with the Oakland Raiders. Over the 10 qualifying seasons, he averaged 0.64 standard deviations better than the mean. His best year was in 2001, when he completed 65.8% of his passes, and only had 1.6% of his attempts intercepted. This was good enough to be 2.54 standard deviations better than the mean that year.

4. Dan Marino (1983, 1st Round, 27th Overall Pick, 6th Quarterback Drafted) - This was the "Year of the Quarterback". Marino was drafted by the Miami Dolphins out of the University of Pittsburgh behind John Elway (1), Todd Blackledge (7), Jim Kelly (14), Tony Eason (15), and Ken O'Brien (24). He started immediately, and went on to throw enough passes in 16 different seasons, averaging 0.51 standard deviations above the mean CMI. His best year was 1984, where he completed 64.2% of his passes, and had 3.0% of his passes intercepted. This was good to finish 1.28 standard deviations above the mean CMI that year. Dan Marino was elected to the Pro Football Hall of Fame in 1995.

3. Brad Johnson (1992, 9th Round, 227th Overall Pick, 13th Quarterback Drafted) - Johnson was drafted by the Minnesota Vikings out of Florida State behind David Klingler (6), Tommy Maddox (25), Matt Blundin (40), Tony Sacca (46), Craig Erickson (86), Casey Weldon (102), Will Furer (107), Chris Hakel (112), Jeff Blake (166), Kent Graham (211), Bucky Richardson (220), and Mike Pawlawski (222). He qualified in 9 different seasons - first with Minnesota (1996-1997), then the Redskins (1999-2000), then Tampa Bay (2001-2003), and finally with the Vikings again (2005-2006). Over those 9 seasons, his CMI averaged 0.73 standard deviations better than the mean. 2005 was his best year, where he completed 62.6% of his passes and only 1.4% of his attempts were intercepted, a performance good enough to be 1.32 standard deviations better than the mean CMI that year.

2. Tom Brady (2000, 6th Round, 199th Overall Pick, 7th Quarterback Drafted) - Brady was selected by the New England Patriots from the University of Michigan behind Chad Pennington (18), Giovanni Carmazzi (65), Chris Redman (75), Tee Martin (163), Marc Bulger (168), and Spergon Wynn (183). After replacing an injured Drew Bledsoe in the second game of the 2001 season, he started every game until he got injured during the first game of the 2008 season. Over the 7 year period from 2001-2007, his CMI was good enough to average 0.82 standard deviations above the mean. His best year came in 2007, when he completed 68.9% of his passes, and was intercepted 1.4% of the time, good enough for 2.14 standard deviations above the mean.  Tom Brady will likely get into the Hall of Fame.

1. Joe Montana (1979, 3rd Round, 82nd Overall Pick, 4th Quarterback Drafted) - Montana was drafted by the San Francisco Forty Niners from the University of Notre Dame behind Jack Thompson (3), Phil Simms (7), and Steve Fuller (23). Although he played every single game in the 1979 season, he only threw 23 passes that year. Beginning in 1980, he qualified every year until 1990, when injuries slowed him down. He was eventually traded to the Kansas City Chiefs, where he played for 2 more seasons. Over those 13 years, his performances on the field, he averaged 1.54 standard deviations above the mean, including the remarkable feat of never having a season below the mean. His best season was 1989, when he completed 70.2% of his passes and had 2.1% of his pass attempts intercepted, a performance so good relative to his peers, that it ranked 2.96 standard deviations above the mean.  Montana was elected to the Pro Football Hall of Fame in 2000.

Now that's a pretty good list!

Quarterbacks and the NFL Draft - Part 3

OK, so we have taken a look at those QBs that were taken first overall over the past 30 years. There have been 13 of them. And we've discussed how they've done, compared to each other. Of course, there have been many quarterbacks taken since 1979 - a total of 340 quarterbacks drafted by NFL teams, not including the 8 quarterbacks that have been taken in supplemental drafts. In this and subsequent posts that follow, we'll take a look at these quarterbacks, and we'll try to answer some questions. Questions such as:

> Do quarterbacks drafted earlier do better than quarterbacks drafted later in a draft?
> Who have been the biggest surprises (performance relative to draft position)?
> Who have been the biggest busts?
> Does it pay to draft a quarterback #1?

Some of these questions are easier to answer than others. In the course of trying to answer these questions, we will encounter some themes we will have to address along the way. Themes such as:

> The economics of the draft
> The goal(s) of a team owner
> The importance of winning

Let's start with a picture of where quarterbacks have been generally drafted over the years.

Selection QBs Taken % of Total 1 13 4% 2-5 15 4% 6-15 18 5% 16-25 13 4% 26-50 25 7% 51-75 22 6% 76-100 40 12% 101-125 26 8% 126-150 25 7% 151-175 31 9% 176-200 27 8% 201-225 31 9% 226-250 24 7% 251-300 19 6% 301-350 11 3% Total 340 100%

Determining whether quarterbacks drafted earlier perform better than quarterbacks drafted later is a little tricky. Here's how I will go about trying to answer this question. First, I will look at how many of these quarterbacks that got drafted "played" in the NFL. I define "played", as the number of seasons a quarterback threw a sufficient amount of passers to have qualified statistically. The statistic that I use (and have used in the past) is the standard deviation relative to the mean as measured by CMI. Next, I will look at how many years, did these quarterbacks play? Certainly, length of playing career could be construed as a measure of success. Third, I will look at the average standard deviation of CMI for these quarterbacks. The higher the average standard deviation, the better the performance.

Let's take a look at the first measure - the likelihood of "playing". Some definitions: "Play" = # of years qualifying as a quarterback. Likelihood of playing is defined as # of players qualifying in at least 1 year relative to # of players drafted.

Here's the data:

Selection QBs Taken QBs "Played" %  "Playing" 1 13 13 100% 2-5 15 14 93% 6-15 18 15 83% 16-25 13 10 77% 26-50 25 15 60% 51-75 22 12 55% 76-100 40 16 40% 101-125 26 8 31% 126-150 25 10 40% 151-175 31 5 16% 176-200 27 8 30% 201-225 31 9 29% 226-250 24 5 21% 251-300 19 2 11% 301-350 11   0% Total 340 142 42%

At least according to just this measure, the evidence is fairly compelling. The results in this table show that players drafted in the earlier rounds are more likely to see significant playing time. It could suggest that the players drafted earlier are in fact more talented, or, if you are skeptical, it might suggest that team management influences coaches to give more playing time to the players drafted earlier to justify their investment (I find this latter suggestion a little hard to believe).

OK, so they are more likely to see more playing time. But what happens when they do play? How do these players perform on the field? I will look at two measures. The first, is the number of years that they play in the NFL. Presumably, the better you are, the longer you get to play (all else being equal). I have actually shown this to be true in previous posts (see here and here). So, if the players drafted earlier are in fact better players, we should see the length of their careers exceeding those of players drafted later. Let's see what the data shows:

Selection QBs Taken QBs "Played" Yrs "Played" 1 13 13 7.2 2-5 15 14 4.7 6-15 18 15 4.2 16-25 13 10 2.6 26-50 25 15 6.9 51-75 22 12 2.8 76-100 40 16 4.0 101-125 26 8 3.1 126-150 25 10 2.5 151-175 31 5 4.6 176-200 27 8 3.3 201-225 31 9 2.7 226-250 24 5 2.6 251-300 19 2 3.0 301-350 11     Total 340 142 4.2

Other than the quarterbacks taken first overall, there does not appear to be a appreciable difference in the # of years played by draft position. I am not suggesting that there isn't. There is. It's just not as much as I would have thought prior to looking at the data.

There are two possible explanations. The first is that, by the time the coaches and scouts have determined those worthy of getting playing time, there should only be minor differences in talent levels as it relates to the draft position. The previous table showed that there is a weeding out process that is more severe at the lower end of the draft, so the ones that do end up getting playing time, are probably on par with quarterbacks taken higher (i.e. pre-season training camp could be used as an additional, more rigorous screening process than merely watching college football games, and the NFL combine). The second reason is that again, there maybe a slight inherent bias to play the early draftees more, simply to give them more opportunities to continue to "prove" themselves.

We're still not done. I said earlier that we'll look at two measures. The first was the length of their careers. The second is their actual statistical performance, as measured by the standard deviation from the mean, using CMI as the measuring statistic. A number greater than zero indicating better than average performance, and a number less than zero indicating worse than average. Let's take a look.

Selection QBs Taken QBs "Played" Yrs "Played" Avg Dev 1 13 13 7.2 -0.06 2-5 15 14 4.7 -0.34 6-15 18 15 4.2 -0.15 16-25 13 10 2.6 0.36 26-50 25 15 6.9 -0.01 51-75 22 12 2.8 -0.32 76-100 40 16 4.0 0.16 101-125 26 8 3.1 -0.01 126-150 25 10 2.5 -0.70 151-175 31 5 4.6 -0.35 176-200 27 8 3.3 -0.16 201-225 31 9 2.7 -0.33 226-250 24 5 2.6 0.42 251-300 19 2 3.0 -0.32 301-350 11       Total 340 142 4.2 -0.11

Looking at this table, it's really difficult to see that the players drafted earlier, who actually ended up playing, have done better than those players drafted later. This suggests to me that once it is determined who actually plays, then the only difference that can be associated with draft order, if anything, is the number of years the player plays, and that probably is somewhat influenced by the team's investment in the players drafted earlier.

So what does this all mean for Matthew Stafford and the Detroit Lions, or, Mark Sanchez and the New York Jets. About the only thing we can be certain of at this point, is that they will play. They may end up playing 5 years, maybe more. As for their actual performance on the field, our best guess is that they'll be about average. Maybe I'm wrong on this. Maybe Matthew Stafford is the next Peyton Manning. Or, he could be the next Tim Couch. Or, he could be the next Eli Manning. Only time will tell.