Saturday, February 14, 2009

Brett Favre - Best Ever?

Brett Favre retired earlier this week.

He played in the NFL for 18 years. He set numerous NFL records during the course of his career. The most impressive has to be the fact that, as a quarterback, he started a record 269 consecutive regular season games. Peyton Manning is second on the list, his streak currently standing at 176 games. Third on the list is Ron Jaworski, his streak having ended at 116 games. Tom Brady is next on the list, but his streak ended in the first game of last season at 110 games. Peyton Manning needs another 94 games - he would have to start every single game through the 14th game of the 2014 season to break Favre's record. In addition to the four mentioned above, only two other quarterbacks in history - Joe Ferguson (107 games), and Dan Marino (95 games) have started that many games consecutively. I suspect Favre's record will never be broken.



People are fascinated with Favre. Many consider him to be among the all-time greatest to ever play the game. Some consider him the greatest ever. I do not belong to either group.


The media loved Favre. And vice versa. He never met a camera he didn't like, always seeking an opportunity to get into the limelight (of course, the media loved that). He also seemed intensely competitive, and the combination is what I believe led him to his unfortunate decision to come back for one more season in 2008.


Let's start from the beginning.


In 1991, the Atlanta Falcons took Favre in the second round of the draft. With Chris Miller as their starting QB, in 1992, the Falcons traded Favre to Green Bay. Favre started the season as the back-up to the 'Magic Man', Don Majkowski. In the third game of the season, against the Cincinnati Bengals, he replaced the injured Majkowski in the first quarter. With 13 seconds left, he threw a 35-yard touchdown that helped the Packers come-from-behind to win 24-23. He started the next game against the Steelers, a game the Packers won. The rest, as they say, is history, as he started every game for the Packers until his trade to the Jets following the 2007 season (for a detailed history of, or, more aptly, a tribute to Favre, see here).


The table below shows how Favre did each year.



Before we go on, some explanations.


In order to be able to truly compare Brett Favre to other quarterbacks so that we can properly gauge his place in history, we can't simply look at the NFL's passer rating system. First, we know that the average NFL passer rating in the NFL has been increasing over time. So, by simply using the NFL's passer rating system, the more recent quarterbacks would appear to have done better than quarterbacks from earlier years (see here and here for evidence). Second, we have to agree that the NFL's passer rating system is an accurate, if not adequate measure of a quarterback's performance. I have previously introduced a metric, CMI, which I believe is a better, if not more understandable measure of performance. CMI, however, also has the same bias as the passer rating system. One way to adjust for, and remove the bias favoring more recent quarterbacks is to look at each year's performance for each quarterback, and relate that to the average performance that year. The measure that we use is the standard deviation. I have discussed previously how using this measure tends to "normalize" performances across years. In other words, when a given year's performance by a particular quarterback is "normalized" to the average for that year, and we do that for every year, we can very easily see how that particular performance compares to other performances across years. Since CMI is very similar to the passer rating system, when using standard deviations, it shouldn't really matter whether we use CMI or the NFL passer rating system (there will be differences, and, I often include both so that you can see the differences). The two graphs below show how using either the "normalized" NFL's passer rating system or "normalized" CMI as measures compare to a standard normal curve with 1,451 observations (that's how many qualified quarterback seasons there have been over the past 69 years).



Ok, so back to the table.


Favre had a "pretty good", or, "above average" first year in 1992. Using the NFL passer rating, he finished 0.57 standard deviations above the average that year. Using a standard normal curve, that would mean between the 71st and 72nd percentiles. If we used CMI as the standard, then his performance that year, 1.24 standard deviations above the mean, would place him between the 89th and 90th percentiles.


He followed that with a sub-par year in 1993, finishing between the 30th and 31st percentiles according to the NFL passer rating system, or between the 35th and 36th percentiles using CMI. Then followed several "above average" to "very good" years (1994-1998), depending on your perspective, followed by a couple of mediocre years (1999-2000), followed by four above average years (2001-2004), followed by his two worst years (2005-2006). He finished his Green Bay tenure with an "above average" year in 2007.


He should have taken the opportunity to retire then.


His last year, his first full season not with Green Bay, was "below average". So he finished his career with 3 "below average" years out of 4. Perhaps the Packers brain trust was on to something when they decided that Aaron Rodgers was their future at the end of the 2007 season.


The two graphical illustrations below show each of Favre's seasons in terms of standard deviations from the mean. In case you haven't noticed, as much as I love statistics, I find that graphs are better way to illustrate a point than a table of statistical measures. I show the CMI graph first and the graph based on NFL passer rating next, for reasons that will become plainly apparent when you read on.





Ok, so until now, we've been looking at Brett Favre's career in a bit of a vacuum. Yes, we have compared his performances to an average, and yes, we have tried to put it into historical perspective, but I haven't given you any 'perspective'.


First, let's compare Brett Favre's career to some individual careers. The biggest problem we face in comparing Favre's career to any other individual career, is obviously the length of his career. There have been only 5 individuals who have thrown enough passes to qualify in at least 15 different seasons. Those are Fran Tarkenton (18 years), Brett Favre (17 years), Dan Marino (16 years), John Elway (16 years), and Johnny Unitas (15 years). So how does Favre's "picture" compare to these quarterbacks, all of whom are in the Hall of Fame?


Fran Tarkenton - a picture of "excellence". With a couple of exceptions, Tarkenton exceeded his peers consistently for a very long time. Quite remarkable. Without question, Favre's picture does not resemble that of Tarkenton.


Dan Marino - Marino was "above average" for many of his years, tailing off towards the end of his career. He was "very good" very early in his career. He really didn't have a "bad" year until his last, and wisely decided to hang it up after that. Although closer in resemblance, Marino's picture still looks better Favre's overall.


John Elway - Elway underperformed his peers much of his early career, exceeding the average in only two of his first ten years. He then was "above average", or even "very good" his last six years. He went out on top, having retired following the Broncos having won back-to-back Super Bowls (Super Bowl XXXII and Super Bowl XXXIII). I think it's safe to suggest that Favre's picture looks better than Elway's.


Johnny Unitas - In my opinion, Favre's picture most closely resembles that of Johnny Unitas. They both started out well, had a couple of mediocre years, then had several years where they outperformed their peers, and ended their careers with a few sub-par years.



What if we compared Favre's average performance (in other words, average each of his year's performance in terms of standard deviations), to other players? Well, over his career, in terms of NFL passer rating, Favre averaged 0.42 standard deviations above the mean (0.26 when looking at CMI). Tarkenton averaged 0.65, Marino averaged 0.63, Elway averaged 0.06, and Unitas averaged 0.51 standard deviations above the mean.


Let's expand the comparison list to players who qualified in at least 10 different seasons. 41 different quarterbacks make that list. The two graphs below illustrate where Favre ranks on those lists.



The same graph as above, except looking at it Using CMI as the measure of performance.




Looking at his career statistically, it is hard to make the argument that he belongs in the truly elite group of all-time quarterbacks - those that would be ranked in the top 3 or top 5 all time. I think it is fair to suggest that Favre has had a very good career. It was certainly a lengthy one, and, in that regard, it was exceptional. But as far as his actual performance on the field, there isn't a measure, or set of measures, that suggest that it was exceptional.


People will suggest that he holds the all-time record for yards, or touchdowns, for example. But that is merely a function of his longevity, just as much as he holds the record for interceptions thrown, all-time.


I also don't discuss measures frequently used as evidence of greatness. MVP awards, for example - these are popularity contests more than a reflection of outstanding statistics or an individual's value to a team (the NFL MVP award is given by the Associated Press). Pro-bowl selections for example are another popularity contest - just look at who got selected this past year in the AFC. Another measure frequently used is Super Bowls. Dan Marino didn't win any Super Bowls. Trent Dilfer won a Super Bowl. Winning percentage is also frequently used - last time I looked, a quarterback doesn't play defense, return kicks or punts, or kick field goals. These pieces of evidence would never enter any serious discussion in terms of statistical evaluation.


My eventual goal is to identify the greatest passer of all time. I will use several statistical measures, many of which are mentioned in this post, to identify the most worthy candidates for consideration. As far as my research is concerned so far, Favre would not make that list. There are several other quarterbacks, not mentioned in this post, nor even displayed in either of the last two graphs that would make more worthy candidates, but that is for a different article.

9 comments:

Anonymous said...

If you're looking for the best passer of all-time, and are not concerned with games won or championships won, then what is the criteria? Stats tell no more about a QB than awards do. In my opinion, it's all about the W. And no one did it better in the modern game than Bart Starr. 5 titles in 7 years doesn't even tell the whole story. Starr had it all: stats, titles, efficiency and leadership. If not Starr then Joe Montana. Take your pick.

KiranR said...

I agree with your assessment that Starr and Montana could be considered among the greatest to ever play the game. From the data that I looked at, specifically, the 10 or more qualifying years, and, subsequently to a handful that qualified in at least 15 different years, it would suggest that Favre would not be on that list. In the graphical illustration of 10 or more years, Bart Starr doesn't show up as he only attempted enough passes to qualify on 8 occasions. But you can quite easily see Montana's greatness. I am waiting to publish my list of greatest passers of all time. If I did it now, I'd have nothing to write about!

Thanks for the feedback.

Anonymous said...

The problem is that there are just way too many factors and variables in trying to find any "best" individual at any position in NFL history. Especially QB. You have incredibly prolific passers like Marino and Fouts that have the numbers but not the titles. Then of course, the cool, efficient field generals like Starr and Montana who have the titles but lesser numbers. And then there are the few that sort of had both (Elway, Unitas, Manning) but also came up short more than a few times in their careers. And how do you compare the bloated passing stats of QBs of the 80s, 90s and today with the "dead ball era" guys when defensive backs were allowed to have their way with receivers? One thing is for certain: There is no, and never will be, a definitive #1 QB of all-time. OK, maybe Montana...

KiranR said...

I agree with you that there are many variables and factors to consider. The beauty of statistical analysis is that you can consider these factors and variables, and adjust for them. We put too much emphasis on a quarterback's contribution to winning or losing. We must remember that he is only one individual on a team of 55 players. He cannot control the offensive line, how the defense plays, whether the field goal kicker is good or bad, or return punts. There are a few things he can control (to a certain degree), and those are things like completing his passes, avoiding interceptions, etc.

As far as your comments go, Marino and Fouts didn't have the #s you think they did - they actually don't stack-up that well when compared to their peers, and subsequently to players of other eras (we do this by using the standard deviation from the mean as the measuring stick). You say that Starr and Montana don't have the #s - actually they do have the #s. And you say that Elway, Unitas and Manning had both. Well, actually, Elway doesn't have the #s that Unitas had, and Manning does have the #s.

You also ask how to compare the "bloated" '80s and '90s #s with the old days. Well, that's where the standard deviation comes in handy. Since we're always comparing players to their peers in a given year, and then stating each statistic (in this case passer rating) in terms of standard deviations from the mean, it shouldn't matter when they played.

I will have to post an article soon on this topic, as it will show who had the #s and who didn't.

I do agree with you that there can't ever really be a definitive #1, but I will attempt, at least statistically to show who might be worthy of that designation. And yes, Montana is worthy of consideration....

RR said...

Did Steve Young not make the cut for your QB shortlist? I admire and respect your effort and enthusiasm.

KiranR said...

Steve Young didn't make the list b/c his career was relatively short compared the players listed here. In another post (see here -
http://newqbrating.blogspot.com/2009/07/was-steve-mcnair-good-passer.html) on Steve McNair, I did include Young, and he could be considered "Best Ever" using 9 years as the minimum criteria.

BILLDPACKERFAN said...

Interesting work, and thanks for sharing, but I have to question the wisdom of dismissing everything not in your vacuum. Does Montana shine as bright without Jerry Rice? Would Peyton's career look so brilliant without Harrison? You bring up how the O-line can help, but fail to factor in that it can also hamper. Same goes with a running game, and a defense. Many, many things factor into how a game is prosecuted, and raw stats will never be able to account for too much. How much better do the guys around Brett Favre play because of his obvious passion? Can that be measured? How many games were lost by second string quarterbacks, while Favre gutted it out EVERY week? Have you attempted to factor in the "Elway Factor"... meaning clutch plays and forth quarter comebacks (Favre has 43 and counting)? I also have to take issue with your dismissal of mobility. While it's true the statisticians record a successful scramble-pass, they offer no more weight than they do the pocket passer with the luxury of picking his spots. Is that the same feat? Does it require equal talent to throw with feet planted in the pocket as it does to toss one across your body while off balance? And what about coaching? Kelly spent most of his career in no-huddle, Peyton chooses plays for himself AFTER the defense is set. And what about running attacks? (Good ones take Safety attention out of the secondary.) Conversely, what effect does the monster passing potential have on taking safeties out of the way of running backs? How many running backs who played with Favre were competitive on other teams? Again, I appreciate your work, but there's a hell of a lot of work left to do before you take the most prolific passer in history of the list of greats.

Tim said...

There are many problems with relying on a statistical approach to determine who the best QB of all time is. I agree it is a useful reference, but there are other important factors or factors that muck up a statistical approach. Let’s look at CMI. It is influenced by factors outside of the QB’s control, primarily other talent on the field: blocking talent, receiving talent, the ability to run the ball and its influence on defensive schemes. It is influenced by offensive game strategy (i.e., good coaching vs. bad coaching) that is not equitable over time. Additionally, the opposing defenses influence these numbers (defensive strength of schedule). Just look at Matt Cassel’s numbers from 2008 and 2009 and you see this “premium” QB’s can gain or lose from switching teams or acquiring or losing talent over time. Or look at the 2008-2009 numbers for Kyle Orton and Jay Cutler. I’d argue that neither team got a better QB – and that Chicago management should stop listening to its fans when evaluating QB talent. How about long term strategy that is so important in the salary cap ear? Here is an example to ponder. If you have Peyton Manning, you'll want to load up on good pass blocking talent even at the expense of run blocking talent, and skimp a bit on the most talented receivers and favor ones with good hands and run good routes. Furthermore, with Manning, you'll have money and draft picks to chase better defensive players. How else does this show up except for the “W” column? For a less talented QB, you might make that QB better by establishing a running attack to force defenses to load up in the “box”, as well as acquire top receiving talent, but this might happen at the expense of defense. How about CMI itself? Although it does capture much of what makes a QB great (e.g., accurate passing, quick decision making, placing the ball in a catchable position at the best speed and angle and tight spin, good field vision to assess open receivers, good communication and teamwork with receivers, reading defenses and even audibling well, etc.) it does not take into account, or wrongly discounts what I'll call good "good game play" (e.g., throwing a deliberate incomplete pass to avoid a sack, throwing an incomplete pass in the two-minute offense instead of a short meaningless pass that would keep the clock running, not throwing to an open short route receiver and taking a safe chance on a long route receiver on a 3rd-and-long down, knowing when an interception is most hurtful or not hurtful at all). These “good game play” decisions hurt the QB’s CMI, but do positively contribute to winning. Furthermore, CMI completely or partially misses many key attributes of a good QB like leadership, clock management, or things like throwing to a receiver in a spot where he has a better chance for yards after the catch. And how about the big omission, the ability to run the ball! I admit, I have a hard time evaluating how important running ability is and its strategic implications, but I enjoy watching teams tinker with speedy QB’s. Finally, I would argue that the QB position has evolved and the skills that are needed have also changed. I laugh at fans who try to speculate what Unitas or Tarkenton would do in today’s game. To that argument, I wonder what Vick and all his speed would have done 30 years ago. You just don’t know! As the skills necessary to excel as QB have evolved, certain stars shine today that might not have years ago and vice versa. The only way to assess these intangibles is to understand the game, watch a lot of games, and understand team strategies. Stats will only get you so far. In that light, and as an avid NFL watcher since the mid-80’s, the best QB’s in my opinion are: John Elway, Peyton Manning, Steve Young, and Brett Favre. Perhaps the most overrated QB’s of all time are Troy Aikman and Kurt Warner (although I’m a huge fan of Kurt!), and the most underrated is Vinny Testaverde. Of course, my own opinion is just that, my own. Enjoy the games, boys!

BabeRuth99 said...

Bart Starr is really overrated he won with no talent in league compared this day and age where 90% of league is running 4.5 40's or better. If Bart Starr played in the NFL of now he get crushed. The game has changed. Joe Montana and John Elway where 2 of the best QB'S to ever play the game. Considering also when the played and the talent that was out there. You just can't compare Bart Starr to Joe Montana Mr.Muggins.