I thought it might be illustrative to show you C scores for all the quarterbacks who qualified in 2009. With Kurt Warner retiring at the end of last season, it is certain that not all of them will qualify again in 2010, and, almost assuredly some of the remaining won't qualify in 2010. The difficulty in looking at these players is their body of work. Not all of them will have a C7 score, our standard measure. That being said however, it is still worth a look. Instead of simply limiting the list to those that have a C7 score, we will look at all of their C scores (C1, C4, C7 and C10).
Here's what the list looks like:
Not too many surprises here.
Did you expect someone other than Peyton Manning to be the best passer in the game today? If you did, I'd love to hear from you, and I'd like to see the analysis. Peyton Manning is one of the all-time best (see here and here), and most certainly the greatest passer of his era. Based on C7, I currently have him 7th all-time (C7 of 644), right behind both Fran Tarkenton and Len Dawson who are tied for sixth at 646. He has an outside chance (I'd say about 25%) of catching Sammy Baugh, who is 5th with a C7 of 654.
In the article I wrote on Kurt Warner (see here), I showed that Warner compares favorably to players like Drew Brees, and Tom Brady. In my recent post formally introducing C scores I suggested that as far as the Hall of Fame, that Warner appears to be right on the fence. I certainly would make the argument that Ken Anderson should be in before Warner.
In that same article I also discussed Tom Brady and Drew Brees. I think if they both played long enough that they would both be in the HOF, and I'm fairly certain that Brees would end up as the better passer.
As far as the next group of players, certainly Carson Palmer and Philip Rivers appear to be on their way to becoming good passers, although they've still got a few years ahead of them before we can make any definitive statements.
And finally, looking further out in to the future, I'd say that Matt Schaub is an up-and-comer, and, really taking a flyer, perhaps Aaron Rodgers can parlay his early success to be considered a great passer. Only time will tell.
Showing posts with label Drew Brees. Show all posts
Showing posts with label Drew Brees. Show all posts
Sunday, May 9, 2010
Tuesday, March 9, 2010
A Look Back At Kurt Warner's Career As a Passer
On the Friday before Superbowl XLIV, Kurt Warner announced that he was retiring from the NFL.
Of course, the conversations now turn to whether he will be enshrined in Canton. There are those who say yes, some who say maybe, and those who say no. The fact is, we don't know. There is no formula to get in.
In many of the posts on the topic, the usual comparisons are made. The easiest comparisons are to his peers - Manning, Brady, Favre, Brees, and others. The other comparisons I've seen are to players like Dan Fouts, Jim Kelly, Troy Aikman, and Warren Moon, all four of whom are in Canton.
My post is not about whether he was a good quarterback or not, and whether he should or should not be in the Hall of Fame. This post is about whether or not he was a good passer. And how did he compare to his peers? And where would he rank in terms of recent quarterbacks who have retired? And how would he compare to all quarterbacks? These are the questions I will explore. All of these questions will be answered in the context of his passing ability in relation to the others.
Ok, let's start with a look back at his performance over the years.
Several things jump out at me.
His career passer rating, for one. It's very high. According to the Pro Football Hall of Fame, Warner's career passer rating of 93.7 ranks fifth best all-time, behind Steve Young, Philip Rivers, Tony Romo and Peyton Manning. As I have discussed before, the NFL passer rating formula isn't a very meaningful way to compare quarterbacks over time. I'll discuss how we adjust the passer rating, and even simplify it, and then compare that figure to his peers, in order to get a more meaningful picture. The average passer rating of 95.1 shown above differs from that shown by the Pro Football Hall of Fame simply because mine is just a simple average of each year.
Another fact that jumps out at me is that he got a late start. He didn't start really playing until he was 28. In my database of 416 quarterbacks going all the way back to 1932, including players from both the AAFC in the late 1940's and the AFL in the 1960's, and only looking at quarterbacks who have qualified in 8 or more years, only 7 quarterbacks had their first qualifying season at age 28 or later. For all quarterbacks who qualified in at least 8 seasons, both the median and mean age at which they qualified in their first season is 24. There are a total of 77 quarterbacks who qualified in at least 8 seasons. Keep in mind that the average number of years in which a quarterback qualifies (given that he qualifies in at least 1 year) is 4.125, and median number of years is 3. So to be one of only 7 quarterbacks who began at age 28 or later and qualify in 8 different seasons is pretty rare company. The other six players are Jeff Garcia and Roger Staubach at age 29, and Warren Moon, Billy Kilmer, Ken Stabler, and Brad Johnson at age 28. Note that Staubach and Moon are both in the Hall of Fame.
Next thing I notice is that there are many gaps in his career. He got injured, then was signed by the NY Giants as an insurance policy against Eli Manning. Following that, he ended up getting an opportunity to play in Arizona, until the Cardinals drafted Matt Leinart. When Leinart got hurt in the fifth game of the 2007 season, Kurt stepped in, and went on to lead the Cardinals into Superbowl XLIII, and into the playoffs this past season, where the Cardinals lost to eventual Superbowl Champs New Orleans.
It is extremely difficult to use the NFL Passer Rating system, especially the career passer rating for a given quarterback to see how they compare to other quarterbacks. Just take a look at the list provided by the Pro Football Hall of Fame. This is simply not a list of the best passers over time. It merely reflects the fact that the passer rating calculation favors current passers. I have previously shown (here) why this is not an accurate reflection of passing ability.
Perhaps one way to adjust the passer rating is to reflect the standards such that the average in each year turns out to be 66.7 (as the intended application of the passer rating). See my post on the subject here. In that regard, Warner's (adjusted) career passer rating is around 83. Of the 416 quarterbacks, that would rank him 23rd. Of the quarterbacks qualifying in 8 or more years, it would rank him 10th.
While the Adjusted Passer Rating is better than the NFL Passer Rating, in that at least one can compare quarterbacks from one era with those of another, it's still not an ideal (or simple) measure of a passer's effectiveness. I've used CMTI in the past, and I'll use it here. More specifically, I'll use the CMTIPR, or percentile rank of CMTI.
In my previous post, I compared Manning, Favre and Brees using CMTIPR. We had to adjust CMTIPR for length of career, and I used their best seasons as the measure. At the end of that post, a graphical illustration showed the effect. I've since improved upon that concept. Using the concept of "Best Seasons", I've come up with 4 statistical measures, C1, C4, C7 and C10, that basically shows a given quarterback's best season (C1), 4 best seasons (C4), 7 best seasons (C7) and 10 best seasons (C10). C1 is simply the quarterback's best season in terms of percentile rank of CMTI. C4 is similarly the sum of the percentile ranks of his 4 best seasons, and so on.
So let's take a look at Warner, and how he compares to Manning, Favre, and Brees.
Obviously, neither Brees nor Warner will have a C10 score as they've not had 10 seasons in which they qualified. I don't think one can evaluate a career based on 1 year, so let's not use C1. It is shown simply for illustration purposes to highlight their best year. So that leaves us with C4 or C7. In order to truly define greatness, one must exemplify superior performance, but also perform at that level for a number of years. The more years performing at a high level, the greater the player. I think that C7 strikes a good balance. In my database, there are 50 quarterbacks who have a C10 score, 95 quarterbacks who have a C7 score, and 169 quarterbacks who have a C4 score. So, at least based on a C7 score, Warner compares very favorably to Manning, Favre and Brees.
We will go into much more details regarding the C scores for quarterbacks in my next post.
Of course, the conversations now turn to whether he will be enshrined in Canton. There are those who say yes, some who say maybe, and those who say no. The fact is, we don't know. There is no formula to get in.
In many of the posts on the topic, the usual comparisons are made. The easiest comparisons are to his peers - Manning, Brady, Favre, Brees, and others. The other comparisons I've seen are to players like Dan Fouts, Jim Kelly, Troy Aikman, and Warren Moon, all four of whom are in Canton.
My post is not about whether he was a good quarterback or not, and whether he should or should not be in the Hall of Fame. This post is about whether or not he was a good passer. And how did he compare to his peers? And where would he rank in terms of recent quarterbacks who have retired? And how would he compare to all quarterbacks? These are the questions I will explore. All of these questions will be answered in the context of his passing ability in relation to the others.
Ok, let's start with a look back at his performance over the years.
Several things jump out at me.
His career passer rating, for one. It's very high. According to the Pro Football Hall of Fame, Warner's career passer rating of 93.7 ranks fifth best all-time, behind Steve Young, Philip Rivers, Tony Romo and Peyton Manning. As I have discussed before, the NFL passer rating formula isn't a very meaningful way to compare quarterbacks over time. I'll discuss how we adjust the passer rating, and even simplify it, and then compare that figure to his peers, in order to get a more meaningful picture. The average passer rating of 95.1 shown above differs from that shown by the Pro Football Hall of Fame simply because mine is just a simple average of each year.
Another fact that jumps out at me is that he got a late start. He didn't start really playing until he was 28. In my database of 416 quarterbacks going all the way back to 1932, including players from both the AAFC in the late 1940's and the AFL in the 1960's, and only looking at quarterbacks who have qualified in 8 or more years, only 7 quarterbacks had their first qualifying season at age 28 or later. For all quarterbacks who qualified in at least 8 seasons, both the median and mean age at which they qualified in their first season is 24. There are a total of 77 quarterbacks who qualified in at least 8 seasons. Keep in mind that the average number of years in which a quarterback qualifies (given that he qualifies in at least 1 year) is 4.125, and median number of years is 3. So to be one of only 7 quarterbacks who began at age 28 or later and qualify in 8 different seasons is pretty rare company. The other six players are Jeff Garcia and Roger Staubach at age 29, and Warren Moon, Billy Kilmer, Ken Stabler, and Brad Johnson at age 28. Note that Staubach and Moon are both in the Hall of Fame.
Next thing I notice is that there are many gaps in his career. He got injured, then was signed by the NY Giants as an insurance policy against Eli Manning. Following that, he ended up getting an opportunity to play in Arizona, until the Cardinals drafted Matt Leinart. When Leinart got hurt in the fifth game of the 2007 season, Kurt stepped in, and went on to lead the Cardinals into Superbowl XLIII, and into the playoffs this past season, where the Cardinals lost to eventual Superbowl Champs New Orleans.
It is extremely difficult to use the NFL Passer Rating system, especially the career passer rating for a given quarterback to see how they compare to other quarterbacks. Just take a look at the list provided by the Pro Football Hall of Fame. This is simply not a list of the best passers over time. It merely reflects the fact that the passer rating calculation favors current passers. I have previously shown (here) why this is not an accurate reflection of passing ability.
Perhaps one way to adjust the passer rating is to reflect the standards such that the average in each year turns out to be 66.7 (as the intended application of the passer rating). See my post on the subject here. In that regard, Warner's (adjusted) career passer rating is around 83. Of the 416 quarterbacks, that would rank him 23rd. Of the quarterbacks qualifying in 8 or more years, it would rank him 10th.
While the Adjusted Passer Rating is better than the NFL Passer Rating, in that at least one can compare quarterbacks from one era with those of another, it's still not an ideal (or simple) measure of a passer's effectiveness. I've used CMTI in the past, and I'll use it here. More specifically, I'll use the CMTIPR, or percentile rank of CMTI.
In my previous post, I compared Manning, Favre and Brees using CMTIPR. We had to adjust CMTIPR for length of career, and I used their best seasons as the measure. At the end of that post, a graphical illustration showed the effect. I've since improved upon that concept. Using the concept of "Best Seasons", I've come up with 4 statistical measures, C1, C4, C7 and C10, that basically shows a given quarterback's best season (C1), 4 best seasons (C4), 7 best seasons (C7) and 10 best seasons (C10). C1 is simply the quarterback's best season in terms of percentile rank of CMTI. C4 is similarly the sum of the percentile ranks of his 4 best seasons, and so on.
So let's take a look at Warner, and how he compares to Manning, Favre, and Brees.
Obviously, neither Brees nor Warner will have a C10 score as they've not had 10 seasons in which they qualified. I don't think one can evaluate a career based on 1 year, so let's not use C1. It is shown simply for illustration purposes to highlight their best year. So that leaves us with C4 or C7. In order to truly define greatness, one must exemplify superior performance, but also perform at that level for a number of years. The more years performing at a high level, the greater the player. I think that C7 strikes a good balance. In my database, there are 50 quarterbacks who have a C10 score, 95 quarterbacks who have a C7 score, and 169 quarterbacks who have a C4 score. So, at least based on a C7 score, Warner compares very favorably to Manning, Favre and Brees.
We will go into much more details regarding the C scores for quarterbacks in my next post.
Sunday, January 31, 2010
Who's Better, Who's Best - Manning, Favre and Brees
Last Sunday we were all treated to some good football. The AFC Championship featured the surprising NY Jets, against the Peyton Manning-led Indianapolis Colts, and the NFC Championship showcased the Drew Brees-led New Orleans Saints who played host to Brett Favre and his Minnesota Vikings.
Not only are these outstanding football teams, but three of the four teams featured three of the game's best quarterbacks in Manning, Favre and Brees. I thought it would be illustrative to take a look at each of the three quarterbacks individually, and then compare the three side by side to get a sense who maybe better, and perhaps even, who may be best. In evaluating their relative merits, I will only consider their ability to throw the football, not win games, or how well they run, or whether they are clutch performers, or how many Super Bowl victories they have.
Here are the career NFL Passer Ratings for these quarterbacks:
Last, we need not assume that the NFL Passer Rating is the measure to be used to evaluate who might be the better passer. The NFL Passer Rating system has a lot of flaws. I will use two measurement methods. The first is the concept of Adjusted Passer Rating. As I have previously mentioned, the average NFL Passer Rating has been steadily increasing every year. The Adjusted Passer Rating normalizes the passer rating so that the average for each year is 66.7. This is then very helpful in evaluating players who played in different years (such as the case here). Last year, I introduced a simple measure of passing efficiency - CMI. I now call this CMTI (or, simplistically, Completions Minus Thrice Interceptions). It is not only simple, but it has a 86% correlation with the NFL Passer Rating. And, because it is so powerful in explaining any variances in the NFL Passer Rating, and because it is so easy to compute, use and explain, I will use it here.
So, to recap, we will make the following adjustments:
Here are the respective #s for each quarterback by season:
Brett Favre
Drew Brees
Peyton Manning
What do these performance measures say? Well, there's a lot of information here, so let's take these one at a time.
The first is that the career averages shown (actually all of the averages shown) are simple arithmetic averages of each of the years, not a true weighted average. You can see that these simple averages are close enough to the true average by looking at the career average NFL passer ratings shown in these three exhibits and compare them to the career average for each of the quarterbacks shown earlier.
The second is that regardless of what measure we use, whether it is the NFL Passer Rating, Adjusted NFL Passer Rating, CMTI, first 8 years, last 12 years or what not, it should be very clear that Peyton Manning has performed at a truly superior level compared to the other 2 players (or anyone else for that matter). Just look at CMTI over the past 8 years, for example. He has averaged being in the 90th percentile over the past 8 years. Basically, that suggests that he is, on average, in the Top 3 every year. Truly remarkable. In Peyton Manning, we are watching one of the all-time best. A once-in-a-generation quarterback. In case you didn't notice, the rows highlighted in green indicate a performance that is in the 85th percentile or higher. The areas in blue are 95th percentile or higher.
Ok, Manning is best. That was easy. Now comes the difficult part. Who's better, Favre or Brees?
The case for Favre:
On an Adjusted Passer Rating basis, Brett Favre's first 8 years are comparable to, if not better than Drew Brees' first 8 years (70th percentile versus 68th percentile). We can't directly compare their two careers since one has had an eighteen year career while the other only eight. It is very evident that, but for the last year, Favre's performance has been quite pedestrian over the past 10 years. Well, we can't necessarily penalize him for that, because we don't know how Brees might do in his next 10 years. We don't even know if he is likely to play another 10 years. Given history, however, it is fairly safe to assume that he won't play another 10 years. In my database, going back to 1932, and including both the AAFC in the 1940's as well as the AFL in the 1960's, I have a total of 416 quarterbacks who qualified in at least 1 year. 206 of these quarterbacks qualified in only 1 or 2 years. The average is 4.125 years. There are only 6 quarterbacks that qualified in 15 or more years. Y.A. Tittle qualified in fifteen different seasons. John Elway, Dan Marino and Johnny Unitas each qualified in sixteen different years. And then there are the ultimate marathon men, Fran Tarkenton and Brett Favre, who both qualified on 18 different occasions.
Michael J. Schell, in his book Baseball's All-Time Best Hitters, makes an adjustment to player batting averages, called a late-career adjustment. In essence, he recognizes that a player's performance declines as he ages, and so he makes an adjustment for it. Schell basically caps the at-bats of a player at 8,000. All at-bats beyond 8,000 are ignored. This way, he can compare each player through their first 8,000 at-bats. Doing that type of adjustment would be extremely difficult in the case of football. We know that the average number of passes attempted by a player per game has increased over time (see here and here), so limiting to a fixed number of attempts would be difficult and meaningless at best.
We could limit the player performances to a given number of seasons. Look at the three graphs below.
The first is the average percentile rank of CMTI for all 416 quarterbacks by age.
And the second is the same as the first, except with the x-axis denoting the number of seasons in which they qualified instead of their age.
It appears that there is a learning curve at first (and an elimination process, as presumably the worst players end up having shorter careers), followed by a decline after about 12 years. Keep in mind that this first graph shows the career percentile rank. What we're really interested in is does performance decline as a player ages? So, what we really want to see is the average percentile rank for a given year of a player's career no matter how long they have played.
This is what the third graph shows. It's actually a bit surprising. There doesn't appear to be a significant drop-off. As a matter of fact, out in the 15th-18th years, their performance seems to have got better! Ok, so in the outer years, we're not talking about a lot of players (for years 17 and 18 its just 2 players - Tarkenton and Favre), so anomalies should be expected. So what do we do?
I mentioned earlier that the average number of years in which a quarterback qualifies is 4.125 for all 416 quarterbacks. The table below shows the data in more detail.
There are a total of 77 quarterbacks who have played (qualified) in 8 or more years. That would be 77/416 = 18.5% of the eligible list. The average number of years for this group is 10.8, and the median number of years for this group is 11. It would seem reasonable then to look at a minimum of 8 years as a cut-off. But how do we recognize players who have played a lot longer? The method that I have chosen is to look at a given player's best 8 years. This means several things. First, the obvious - it eliminates from consideration any player who has qualified in fewer than 8 years. Second, it accounts for the fact that players have played longer, and, that fact doesn't hurt them. As a matter of fact, it's the opposite. The less a player has played, the more the "penalty". Third, given the small sample of players playing beyond 14 years, it is quite possible that one or more of them had good years in those later years, so why exclude those years?
So, going back to the tables where I showed each of the three quarterbacks and their individual seasons, you can see that Favre's best 8 years are better than Brees' 8 years. This gives credit to Favre's lengthy career. Remember, less than 20% of the qualified quarterbacks even get to 8 years total, let alone be fortunate enough to have 8 good years. Perhaps over time, Brees might very well exceed Favre. And, if and when he does, we'll give him credit for it at that time.
The case for Brees:
Although comparable to Favre in his first 8 years based on Adjusted Passer Rating, using CMTI gives Brees the nod. Also notice that Brees has had four years where he was better than 85th percentile, while Favre only had one. Brees' last four years in New Orleans have been especially strong. In each of years 5-8, Brees outperforms Favre, while in the first four, Favre outdoes Brees 3 out of four years. Favre, it appears, built his legend largely on the success of his first year, and perhaps even starting with his first game. Check out the following chart.
It shows the average percentile rank for all quarterbacks who played longer than 8 years, for each year they played. Actually, it's not quite the number of years they played in as much as it is the number of years in which they threw enough passes to qualify to have a rating. It's the blue line, and it includes data from the previously mentioned group of 77 quarterbacks. The blue line in this graph is different than the line shown on the third graph earlier, in that this only includes data for those quarterbacks qualifying in 8 or more years. The red line is the data for Drew Brees. The green line is the data for Favre. Clearly, outside of his first year, and then his 3 MVP years (years 3-5 for him), Favre hugged the average for this group of quarterbacks. Some years he did better, and some he did worse.
We're almost done. One more graph.
This one puts a slight twist on the notion of the best years. Instead of just looking at the best 8 years, why don't we look at the best n years, for each year n that the quarterback has played. So this way, we can look at the best year for each quarterback, the best two years, the best three years, and so on. For good measure, I also threw in Manning's #s.
So what do we conclude from all of this?
Again, if there, was any doubt before, this should put it to rest. Manning is clearly the best.
Between Favre and Brees, an argument could be made for either. In terms of a total career, Favre gets the nod, simply because of the number of years he's played. Brees has put up some impressive numbers so far. If he continues this for a few more years, he will most likely finish ahead of Favre, but that remains to be seen. Based on the evidence so far however, he is doing at least as well as, if not better than Favre did in his first 8 years. Perhaps we'll check-in in a few years when Brees retires. And, speaking of retirement, in my next post, I'll take a look at Kurt Warner's career.
Not only are these outstanding football teams, but three of the four teams featured three of the game's best quarterbacks in Manning, Favre and Brees. I thought it would be illustrative to take a look at each of the three quarterbacks individually, and then compare the three side by side to get a sense who maybe better, and perhaps even, who may be best. In evaluating their relative merits, I will only consider their ability to throw the football, not win games, or how well they run, or whether they are clutch performers, or how many Super Bowl victories they have.
Here are the career NFL Passer Ratings for these quarterbacks:
Manning is clearly the best. Well, at least that's what looking at the career NFL Passer Rating will tell you. These figures differ from what you would find at the Pro Football Hall of Fame (as of January 23rd, 2010). The reason being that the figures listed on the Pro Football Hall of Fame site were as of the beginning of the 2009 season, and mine are as of the end of the 2009 season just completed.
For reasons that I have chronicled in the past (see here, here, and here), the NFL Passer Rating formula is in and of itself is of little use in determining who among the three might be best. The three quarterbacks very well might end up being ranked this way when it is all said and done, but that doesn't justify the rating system.
In order to complete our analysis, we have to make several adjustments to the NFL's passer rating system.
The first, is that we need to take each player's passer rating, and relate it to some baseline, so that we may account for players playing in different eras (yeah, I know, it's a little bit odd to be talking about players in different eras when comparing three current quarterbacks, but Brett Favre has been throwing since 1992, and Drew Brees only since 2002. And since the average NFL passer rating has been increasing over time (see here), we need to make an adjustment for that. The method that I have chosen is to relate a given quarterback's performance relative to the mean for that year, and use the standard deviation in that year as the measuring stick. In other words, the difference between the quarterback's actual performance compared to the mean, as a function of the standard deviation. As I described here, we then convert each of these resulting measures, also called z-values, into percentile ranks, using the standard normal distribution.
The second, is that we need to account for the fact that they've each played a different number of seasons. While I used a rather simplistic approach last year, I will present a more sophisticated approach starting with this blog. This method will need further refinement over time, since I am early in my thought process regarding this particular topic. However, it is an improvement over last year's simplistic approach. Actually, the method used last year (average deviation) really doesn't take into account the number of years at all. When I evaluated Brett Favre's career, I simply compared him to a group of quarterbacks who had played a like number of years, and simply ignored any comparisons to players who played far fewer seasons. That method won't work in this case, as we need to compare three quarterbacks with three vastly different career lengths. All else being equal, and they rarely, if ever, are, a player who has a 10 year career versus a player who has a 7 year career should get a higher rating (even if, and especially if, they both had the exact same ratings in every year they played). But the question becomes, "how much higher"?
The third, is that we need to somehow account for the fact that each of the three quarterbacks has played not only a different number of seasons, but that those seasons represent different parts of their careers. In some respects, since the last 8 seasons have been common to all three, we could simply look at the last 8 seasons. But, that would mean that we would be comparing Brett Favre in his 11th through 18th seasons, Peyton Manning through his 5th through 12th seasons, and Drew Brees his first 8 seasons. As I have shown in the past, the early years for a quarterback tend to be the "learning years", and hence this comparison would be inappropriate without the proper adjustments. Brett Favre is also 40 years old, and we have seen declining performances as quarterbacks age (actually, their performances increase at first as they gain experience, and then subsequently decline as the physical toll takes effect). One method is to look at each player's first 8 years, since in that case the comparisons would be over the identical portions of their careers. And perhaps we could compare Favre and Manning for the first 12 years.
Last, we need not assume that the NFL Passer Rating is the measure to be used to evaluate who might be the better passer. The NFL Passer Rating system has a lot of flaws. I will use two measurement methods. The first is the concept of Adjusted Passer Rating. As I have previously mentioned, the average NFL Passer Rating has been steadily increasing every year. The Adjusted Passer Rating normalizes the passer rating so that the average for each year is 66.7. This is then very helpful in evaluating players who played in different years (such as the case here). Last year, I introduced a simple measure of passing efficiency - CMI. I now call this CMTI (or, simplistically, Completions Minus Thrice Interceptions). It is not only simple, but it has a 86% correlation with the NFL Passer Rating. And, because it is so powerful in explaining any variances in the NFL Passer Rating, and because it is so easy to compute, use and explain, I will use it here.
So, to recap, we will make the following adjustments:
- Relate performance in given year to the mean, and then in relation to standard deviation for that year - then convert to percentile rank
- Use a 'better' measure than NFL Passer Rating. We will show that Adjusted NFL Passer Rating is 'better', and CMTI is simpler
- Take into account differing number of seasons played by each quarterback
- Take into account quarterbacks who are at different stages of their career
Here are the respective #s for each quarterback by season:
Brett Favre
Drew Brees
Peyton Manning
What do these performance measures say? Well, there's a lot of information here, so let's take these one at a time.
The first is that the career averages shown (actually all of the averages shown) are simple arithmetic averages of each of the years, not a true weighted average. You can see that these simple averages are close enough to the true average by looking at the career average NFL passer ratings shown in these three exhibits and compare them to the career average for each of the quarterbacks shown earlier.
The second is that regardless of what measure we use, whether it is the NFL Passer Rating, Adjusted NFL Passer Rating, CMTI, first 8 years, last 12 years or what not, it should be very clear that Peyton Manning has performed at a truly superior level compared to the other 2 players (or anyone else for that matter). Just look at CMTI over the past 8 years, for example. He has averaged being in the 90th percentile over the past 8 years. Basically, that suggests that he is, on average, in the Top 3 every year. Truly remarkable. In Peyton Manning, we are watching one of the all-time best. A once-in-a-generation quarterback. In case you didn't notice, the rows highlighted in green indicate a performance that is in the 85th percentile or higher. The areas in blue are 95th percentile or higher.
Ok, Manning is best. That was easy. Now comes the difficult part. Who's better, Favre or Brees?
The case for Favre:
On an Adjusted Passer Rating basis, Brett Favre's first 8 years are comparable to, if not better than Drew Brees' first 8 years (70th percentile versus 68th percentile). We can't directly compare their two careers since one has had an eighteen year career while the other only eight. It is very evident that, but for the last year, Favre's performance has been quite pedestrian over the past 10 years. Well, we can't necessarily penalize him for that, because we don't know how Brees might do in his next 10 years. We don't even know if he is likely to play another 10 years. Given history, however, it is fairly safe to assume that he won't play another 10 years. In my database, going back to 1932, and including both the AAFC in the 1940's as well as the AFL in the 1960's, I have a total of 416 quarterbacks who qualified in at least 1 year. 206 of these quarterbacks qualified in only 1 or 2 years. The average is 4.125 years. There are only 6 quarterbacks that qualified in 15 or more years. Y.A. Tittle qualified in fifteen different seasons. John Elway, Dan Marino and Johnny Unitas each qualified in sixteen different years. And then there are the ultimate marathon men, Fran Tarkenton and Brett Favre, who both qualified on 18 different occasions.
Michael J. Schell, in his book Baseball's All-Time Best Hitters, makes an adjustment to player batting averages, called a late-career adjustment. In essence, he recognizes that a player's performance declines as he ages, and so he makes an adjustment for it. Schell basically caps the at-bats of a player at 8,000. All at-bats beyond 8,000 are ignored. This way, he can compare each player through their first 8,000 at-bats. Doing that type of adjustment would be extremely difficult in the case of football. We know that the average number of passes attempted by a player per game has increased over time (see here and here), so limiting to a fixed number of attempts would be difficult and meaningless at best.
We could limit the player performances to a given number of seasons. Look at the three graphs below.
The first is the average percentile rank of CMTI for all 416 quarterbacks by age.
And the second is the same as the first, except with the x-axis denoting the number of seasons in which they qualified instead of their age.
It appears that there is a learning curve at first (and an elimination process, as presumably the worst players end up having shorter careers), followed by a decline after about 12 years. Keep in mind that this first graph shows the career percentile rank. What we're really interested in is does performance decline as a player ages? So, what we really want to see is the average percentile rank for a given year of a player's career no matter how long they have played.
This is what the third graph shows. It's actually a bit surprising. There doesn't appear to be a significant drop-off. As a matter of fact, out in the 15th-18th years, their performance seems to have got better! Ok, so in the outer years, we're not talking about a lot of players (for years 17 and 18 its just 2 players - Tarkenton and Favre), so anomalies should be expected. So what do we do?
I mentioned earlier that the average number of years in which a quarterback qualifies is 4.125 for all 416 quarterbacks. The table below shows the data in more detail.
There are a total of 77 quarterbacks who have played (qualified) in 8 or more years. That would be 77/416 = 18.5% of the eligible list. The average number of years for this group is 10.8, and the median number of years for this group is 11. It would seem reasonable then to look at a minimum of 8 years as a cut-off. But how do we recognize players who have played a lot longer? The method that I have chosen is to look at a given player's best 8 years. This means several things. First, the obvious - it eliminates from consideration any player who has qualified in fewer than 8 years. Second, it accounts for the fact that players have played longer, and, that fact doesn't hurt them. As a matter of fact, it's the opposite. The less a player has played, the more the "penalty". Third, given the small sample of players playing beyond 14 years, it is quite possible that one or more of them had good years in those later years, so why exclude those years?
So, going back to the tables where I showed each of the three quarterbacks and their individual seasons, you can see that Favre's best 8 years are better than Brees' 8 years. This gives credit to Favre's lengthy career. Remember, less than 20% of the qualified quarterbacks even get to 8 years total, let alone be fortunate enough to have 8 good years. Perhaps over time, Brees might very well exceed Favre. And, if and when he does, we'll give him credit for it at that time.
The case for Brees:
Although comparable to Favre in his first 8 years based on Adjusted Passer Rating, using CMTI gives Brees the nod. Also notice that Brees has had four years where he was better than 85th percentile, while Favre only had one. Brees' last four years in New Orleans have been especially strong. In each of years 5-8, Brees outperforms Favre, while in the first four, Favre outdoes Brees 3 out of four years. Favre, it appears, built his legend largely on the success of his first year, and perhaps even starting with his first game. Check out the following chart.
It shows the average percentile rank for all quarterbacks who played longer than 8 years, for each year they played. Actually, it's not quite the number of years they played in as much as it is the number of years in which they threw enough passes to qualify to have a rating. It's the blue line, and it includes data from the previously mentioned group of 77 quarterbacks. The blue line in this graph is different than the line shown on the third graph earlier, in that this only includes data for those quarterbacks qualifying in 8 or more years. The red line is the data for Drew Brees. The green line is the data for Favre. Clearly, outside of his first year, and then his 3 MVP years (years 3-5 for him), Favre hugged the average for this group of quarterbacks. Some years he did better, and some he did worse.
We're almost done. One more graph.
This one puts a slight twist on the notion of the best years. Instead of just looking at the best 8 years, why don't we look at the best n years, for each year n that the quarterback has played. So this way, we can look at the best year for each quarterback, the best two years, the best three years, and so on. For good measure, I also threw in Manning's #s.
So what do we conclude from all of this?
Again, if there, was any doubt before, this should put it to rest. Manning is clearly the best.
Between Favre and Brees, an argument could be made for either. In terms of a total career, Favre gets the nod, simply because of the number of years he's played. Brees has put up some impressive numbers so far. If he continues this for a few more years, he will most likely finish ahead of Favre, but that remains to be seen. Based on the evidence so far however, he is doing at least as well as, if not better than Favre did in his first 8 years. Perhaps we'll check-in in a few years when Brees retires. And, speaking of retirement, in my next post, I'll take a look at Kurt Warner's career.
Tuesday, January 6, 2009
Stat of the Week - #1
I'm introducing a new feature - Stat of the Week - little or no commentary - just a stat - mostly because I'm not sure I'll be able to post a blog every week because a) I love my day job, and that takes a lot of my time, and b) posting, especially an interesting, well-researched post, takes time, and I'm just not sure I have the time to do a detailed post every week. So enjoy....
Labels:
2008 NFL Season,
Dan Marino,
Drew Bledsoe,
Drew Brees,
NFL,
NFL History,
Passing Attempts
Saturday, December 20, 2008
Debating the 2009 Pro-Bowl Selections
Earlier this week, the NFL announced the rosters for the 2009 Pro Bowl. Representing the AFC are QBs Peyton Manning, making his ninth appearance, Jay Cutler, making his first appearance, and Brett Favre, making his tenth appearance, but first time representing the AFC. Representing the NFC are QBs Kurt Warner, making his fourth appearance, and Drew Brees and Eli Manning, who is making his first appearance. This will be the first time that brothers playing quarterback have been selected to the Pro Bowl.
I am really not sure what criteria the voters (the players) use to select these quarterbacks. Some probably use statistics, such as the quarterback rating, even though no one actually understands the QB rating or how it is derived. Some probably look at the team's performance, even though the quarterback may have little to do with the team's actual performance, or won-loss record. And then some may simply be basing it on some other factors, such as like-ability, or popularity. Based on the selections, it appears that voters reward players for a good team record and # of touchdowns thrown (to the average fan, these two factors seem like reasonable criteria, I'm sure). I of course disagree, but then again, that is the reason for this post.
The Pro Bowl is meant to be a reward for performing exceptionally well on the field, and, since it is voted on by the players, a recognition from one's teammates as well as competitors. So how do these selections stack up?
Before I get into the details, a quick refresher on the NFL passer rating calculation. I'll keep the math simple. The NFL passer rating is made up of four different components (see here for the details), each with a maximum possible value of 2.375, and a minimum possible value of 0. The four components are given equal weight in the calculation of the rating. The four components are:
Pass Completion percentage
Yards per Completion
TDs per Attempt (Touchdown percentage)
INTs per Attempt (Interception percentage)
In the table below, I show how each of the quarterbacks scored on each component, how that translated to their QB rating, and their respective ranks (amongst each of the four components as well as the final QB rating). I also include CMI for each quarterback, and I have previously discussed my preference for CMI versus the current NFL passer rating formula. The players shaded in green were the ones voted in. The players shaded in blue are my obvious snubs, and the players shaded in orange one could argue deserve to go.
Ok, now on to the #s.
AFC
Peyton Manning - As I mentioned in a previous post, he is having another great year. He has a passer rating of 90.3 (through 14 games). In the AFC, he ranks 5th, 7th, 4th and 6th in each of the components, good enough for 4th in terms of passer rating. He is 3rd in the AFC in terms of # of touchdowns passes. He has led the Colts to a 10-4 record. He is 2nd in the AFC in terms of CMI. Very difficult to argue against the selection.
Brett Favre - For the four components, he is ranked 1st, 9th, 2nd and 16th, good enough for 7th overall in passer rating in the AFC. He has taken the Jets to a 9-5 record, and he has thrown 21 TDs, fourth amongst AFC quarterbacks. He has also thrown 17 INTs, worst in the NFL. He ranks 7th in the AFC in CMI. Based on statistics, should not have made it. The only explanation is that he is probably perceived to have helped turn around the Jets, and he has thrown a lot of touchdowns.
Jay Cutler - Jay Cutler got off to a red-hot start, and has since cooled off. He ranks 9th, 4th, 3rd and 11th on each of the four components, placing him 5th in the AFC in Passer Rating. The Broncos are 8-6, and his 24 touchdowns are second only to Philip Rivers' 28. His CMI places him 9th in the AFC. Based on statistics, again hard to see why he got voted into the Pro Bowl.
NFC
Kurt Warner - He led the Cardinals to a 8-6 record and the NFC West Division title. He ranks 1st, 4th, 4th and 7th in each of the components, good enough for 2nd in the NFC in passer rating. He has thrown 26 touchdowns, 2nd best in the NFC. He ranks 2nd in CMI in the NFC. Based on the #s, good enough to go.
Drew Brees - Started out blazing, but has had a couple of mediocre games recently. Has led the Saints to a 7-7 record, and has thrown a league-high 28 touchdowns. He has also thrown an NFC-high 16 interceptions. He ranks 3rd, 3rd, 2nd and 10th in each of the components, good enough for 4th in the NFC in terms of passer rating. He ranks 4th in CMI. An acceptable choice for the Pro Bowl.
Eli Manning - He has led the NY Giants to an 11-3 record, coming off that Giant upset in last year's Super Bowl against the previously undefeated Pats. He has thrown 20 touchdown passes, only 6th-best in the NFC. His ranks in the four components are 10th, 11th, 5th and 5th respectively - making his passer rating only 9th-best in the NFC. His CMI rank is 10th. Clearly a sentimental pick here. I would not have voted him in.
In summary,
Peyton Manning - In
Brett Favre - Out
Jay Cutler - Out
Kurt Warner - In
Drew Brees - In
Eli Manning - Out
Obvious Snubs
There are two, in my mind - one each in the AFC and NFC, respectively.
In the AFC, Chad Pennington is an obvious snub. Clearly, he doesn't get attention amongst his peers, because he doesn't throw a lot of touchdowns. I've never understood why throwing a touchdown is so important to determining how good a passer someone is. CMI, which has a 97% correlation with the current NFL passer rating, doesn't include touchdowns. Pennington has thrown for only 14 touchdowns. He has led the Miami Dolphins to a 9-5 record (keep in mind that Miami only won 1 game last season). Based on the four components, he ranks 3rd, 3rd, 10th (the component associated with TDs), and 1st, good enough for second-best in the AFC in passer rating. He ranks 1st in CMI.
In the NFC, Jeff Garcia is an obvious snub. Although he has led the Bucs to a 9-5 record, he suffers from the same 'disease' that Pennington has. He doesn't get a lot of recognition around the league for how good he is because he doesn't throw a lot of touchdowns. He has 'only' thrown 10 touchdowns. That hurts him in terms of passer rating, as he ranks, 2nd, 7th, 12th and 1st in each of the components, good for 3rd in the NFC in terms of passer rating. He ranks #1 in the NFC in CMI.
Possibly Overlooked
In the AFC, both Matt Schaub and Philip Rivers have possibly been overlooked. Matt Schaub has led the Texans to a 7-7 record, and has a passer rating of 92.7, 3rd-best in the AFC. In terms of the four components, he ranks 2nd, 2nd, 6th and 13th, respectively. He is 3rd in the AFC in CMI. Philip Rivers, although having a good year (in terms of the traditional touchdowns thrown), suffers from the Chargers being 6-8. He leads the NFL in passer rating at 101.4, and leads the NFL in touchdowns thrown with 28. He ranks 6th, 1st, 1st and 7th in each of the components, and ranks 4th in the AFC in CMI.
In the NFC, Jason Campbell and Aaron Rodgers have both been overlooked in my opinion. Campbell's Redskins are 7-7 on the season, and he has thrown 'only' 12 touchdowns. He ranks 7th, 12th, 13th, and 2nd in each of the four components, good enough for only 10th in terms of passer rating. He ranks 3rd in CMI. What hurts Rodgers most is the fact that the Packers are 5-9. He ranks 4th in the NFC in touchdowns thrown, with 23. He ranks 4th, 6th, 3rd and 9th in the four components, good for 6th in the NFC in passer rating. His CMI rank is also 6th.
In looking at all these quarterbacks, something struck me, and that is the exceptional seasons that Garcia, Campbell and Pennington are having in terms of the infrequency of interceptions thrown (see here for their stats). I'll have a post on this later.
It appears to me that what gets attention (and votes) is winning and throwing touchdowns. At some point I will do an extensive analysis showing the relationship between a quarterback and winning. What doesn't get attention (but should) is interceptions thrown.
The table below shows the stats for the NFL's quarterbacks through 14 games this season (only those who 'qualified' (i.e. those throwing at least 224 passes) are shown.

Subscribe to:
Posts (Atom)











