Better Than Jay Cutler

A lot of noise this past month about Jay Cutler. First, the Denver Broncos' new head coach, former New England Patriots offensive coordinator Josh McDaniels, apparently tried to work a trade to get Matt Cassel from the New England Patriots. This apparently didn't it well with the Pro-Bowl quarterback. After a lot of posturing from both sides, Cutler was eventually traded to the Chicago Bears for Kyle Orton and draft picks. This post isn't about who was right or wrong in the fracas, or which team got the better end of the deal in the trade. This post is about answering the following two questions:

1) How good is Jay Cutler?

2) Are there quarterbacks with similar experience that are better than Jay Cutler, and, if so, who are they?

The table below ranks each of the 30 quarterbacks that threw enough passes last year to qualify to be ranked. The ranking is based on CMI, and not the NFL's quarterback rating system. In numerous posts prior to this, I have discussed why CMI is a better measure of a quarterback's passing performance than the NFL passer rating system.

Just looking at last year, the year that Jay Cutler got named to his first Pro-Bowl, he was, by this measure, a below-average quarterback. His CMI of 0.536 was (0.14) standard deviations below the mean. It should be noted that even though he was below average relative to all quarterbacks in 2008, it wasn't unexpected. Previously, I discussed the notion of performance improving with experience. So, if you were to look only at quarterbacks and their second year performance, their average is (0.21) standard deviations relative to the mean. Relative to that measure, he actually did better than the typical second year quarterback.

Let's take a closer look at the table above. The last 2 columns indicate the # of years the quarterback has played since he first qualified, and the total # of years he has qualified. So, Cutler has played 2 seasons where he threw enough passes to qualify, and this is second consecutive year of being qualified. There are six other quarterbacks in this table who have similar experience.

They are (rank in parenthesis):

Jason Campbell - WAS (5)

Matt Schaub - HOU (7)

Trent Edwards - BUF (8)

David Garrard - JAX (12)

Jay Cutler - DEN (16)

Derek Anderson - CLE (30)

Of the six quarterbacks who were in their second year (Garrard has the most experience of all the quarterbacks, even though he has only qualified in two years), Cutler only outperformed Derek Anderson of Cleveland in 2008. Ok, so 1 year does not make a quarterback. Let's take a look at these six quarterbacks and see how they've done in each of their first two (qualifying) years.

According to this table, both Garrard and Schaub have outperformed Cutler in each of 2007 and 2008. And, arguably, over the two-year period, so did Campbell. The only quarterback in this class that has clearly underperformed the rest of the group is Derek Anderson. He will not last long in the NFL.

Admittedly, it's difficult to judge a quarterback after his first two years. But, so far at least, there appear to be several quarterbacks that have outperformed Cutler. It will be interesting to watch this group over the next several years, and especially Cutler, as he will be with a new team.

The Best and Worst of 2008

This is a simple post.  Really simple.  The list should pretty much speak for itself.  The best and worst of 2008.  I use the same criteria I've been using the past few posts.  CMI, and, more specifically, standard deviation of a player's CMI relative to the mean.  30 players attempted enough passes in 2008 to make the list.  

The Best 3:

3.  Jeff Garcia, Tampa Bay Buccaneers - Why no team has picked him up is a complete mystery to me - the man can still play, even though his durability is certainly a question.  Completed nearly 65% of his passes, and less than 1.6% of his passes were intercepted.

2.  Peyton Manning, Indianapolis Colts - Do we even need to say anything? Coming off off-season knee surgery, Mr. Consistency now holds the highest current active streak of games started consecutively - 176 games. He completed almost 67% of his passes.

1.  Chad Pennington, Miami Dolphins - Let go by the Jets so that they could welcome Brett Favre (that worked out well for the Jets, didn't it?) and subsequently picked up by the Dolphins. Had a terrific season, completing more than 67% of his passes, and less than 1.5% of his passes were intercepted.

In previous posts on my blog, I have spent time discussing each of the above 3 players.

The Worst 3:

28.  Tyler Thigpen, Kansas City Chiefs - Won't show up at the bottom of the list in 2009.  Why?  Matt Cassel (#10 in 2008).  The Chiefs made a deal with the New England Patriots to get Cassel (and Mike Vrabel) for a 2nd round pick in the upcoming draft.  In 2008, Thigpen, replacing Damon Huard, completed fewer than 55% of his passes.

29.  Gus Frerotte, Minnesota Vikings - Replaced an ineffective Tarvaris Jackson in the 3rd game of the season, started 11 games, and then was himself replaced by Jackson against the Lions.  Completed less than 60% of his passes, and at almost 5.0%, was one of only three quarterbacks whose interception percentage exceeded 3.0% - the other two being Ben Roethlisberger of the Steelers (3.2%), and Brett Favre of the Jets (4.2%).

30.  Derek Anderson, Cleveland Browns - Barely completed 50% of his passes, then got hurt.  Was replaced by Brady Quinn.  Is it Quinn's time in 2009?

Here's the entire list.

My next post - a pre-season ranking for 2009

First Year Quarterback Performance - An Evaluation Method

This past weekend, the New England Patriots traded Matt Cassel to the Kansas City Chiefs.  In fact, they traded linebacker Mike Vrabel and Cassel for a 2nd round pick (pick #34) in the upcoming April draft (see here for the details).  Depending on what your news source is, this was labeled as either a good deal for Kansas City, or a bad deal for New England.  The experts discussing this issue don't seem to support their arguments by any objective measures - they just opine.  I am not going to jump into the debate here.  I will offer this however - we will probably not know who will end up on the better end of this deal until many years from now, and even then, we may not know.  

What about the objective evidence?

Matt Cassel is not the only quarterback in 2008 that had a good year as a first-year quarterback.  There was Aaron Rodgers of the Packers, Shaun Hill of the 49'ers, and Matt Ryan of the Falcons.  There were four other quarterbacks who threw enough passes this year to make the list of first-year quarterbacks - Ryan Fitzpatrick of the Bengals, Joe Flacco of the Ravens, JaMarcus Russell of the Raiders and Tyler Thigpen of the Chiefs.  

Notice here that I didn't use the term rookie quarterback, but instead used the term first year.  The way I look at a quarterback, is that I only consider a quarterback who has thrown "enough" passes to qualify in a given year.  The # of passes has varied by year (as opposed to the fixed standard used by NFL.com).  For the 69-year period from 1940-2008, my database contains 1,451 passing seasons, 336 of which were by a quarterback qualifying for the first time.

How did the aforementioned 8 quarterbacks do?  

I use a statistical measure I have previously developed and discussed, called CMI (Completions Minus Interceptions, calculated as Completions/Attempts - 3*Interceptions/Attempts) instead of the NFL Passer Rating system (using the NFL passer rating system gets you essentially the same answers), and then use the standard deviation from the mean as the measuring stick (for the pool of qualifying passers, I calculate the the mean and standard deviations for each year, and relate a particular performance in that year to the mean using that year's standard deviation).  Using the mean and standard deviation this way not only allows me to compare these 8 quarterbacks relative to each other, but also relative to all quarterbacks this year, or any other year.

In a post not too long ago, I showed how, over the past 69 years from 1940-2008, when the standard deviation is used as the measuring stick, the accumulated data form a near-perfect theoretical standard normal distribution.  And I also showed how, this is true regardless of whether one used the NFL passer rating system or CMI as the statistical base.

I can now show you how these first year quarterbacks did compare to the rest of the QBs in 2008, using this measure.  But this picture may not tell us much, if anything.  Here's the graphical illustration:

Perhaps the only thing that may be apparent from this is that first year quarterbacks don't do nearly as well as quarterbacks with more experience.  Even drawing that conclusion from this is a little iffy.  However, it is clear (at least by this measure) that Cassel had the best year of the 8 first year QBs.  Rodgers was a close second.  JaMarcus Russell and Tyler Thigpen had disastrous first years.  In an earlier post regarding Jeff Garcia, I had suggested that Kansas City might be a good place for Garcia.  Scott Pioli, the new Chiefs' GM apparently was thinking the same thing - upgrade at quarterback, and he did - with Cassel.

Going back to the standard normal curve, let's look at just the first year performances.  Recall that, when all 1,451 seasons are aggregated in a histogram, it looks very much like a standard normal curve.

This doesn't quite look "normal", does it?  The first year performances appear skewed to the left, or, compared to all quarterback performances, appear to under-perform.  Since the total database is normally distributed, then performances in years 2 and beyond must obviously be skewed to the right.  This prompts the question: does experience improve performance, and, in particular, how and when?

One way to look at this is to take the entire database, and look at the average standard deviation from the mean for each additional year a player qualified.  Here's what that graph looks like.

Wow!  I didn't realize that it takes about 5 qualifying years to turn into an average quarterback.  And that's just for those playing that long!  Even though this graph goes out to 18 years, there is hardly any data beyond the 15th year (there are only 8 quarterbacks who have had qualifying seasons 16 different years).  According to this illustration, basically, first year quarterbacks, as a group, struggle mightily.  They then gradually improve each year until about year nine.  Quarterbacks, on average, must still endure sub-par years in years 2-4, and for those that survive, their reward is about four more years of improving performance, assuming they stay healthy for that long.

Note: This analysis - the method of breaking down a quarterback's expected career in terms of passing performance as it relates to experience and how it is expected to change over time, is the first I've seen published publicly.  But I digress.

Ok, so we've taken a look at these 8 QBs relative to all quarterbacks in 2008, and we've taken a look at first year quarterbacks in general.  So the next step is to take a look at the best first year seasons.  In my database, there are 336 first year seasons in all (in other words, in the 69-year period from 1940-2008, 336 different quarterbacks threw enough passes in a season to qualify at least once).  The table below shows the 75 best.

Cassel (#48) and Rodgers (#60) are the only 2 of the 8 in 2008 that crack the top 75 all-time.  On the other end of the scale, we get JaMarcus Russell (#264) and Tyler Thigpen (#290).

When Matt Ryan (#128) got off to a solid start, and the Falcons surprised many by getting into the playoffs, there were many debates as to whether his season was one of the best ever by a first year player.  Not according to this measure.  Often, during those discussions, Kurt Warner's first year was brought up.  In looking at the table above, well, he did have an outstanding first year in 1999.  It ranks #6 all-time in terms of first year seasons.  The best ever - Roger Staubach's 1971 season (Staubach was a rookie in 1966, and played in both 1969 and 1970.  However, he didn't attempt enough passes in either year to qualify).  

Notable quarterbacks high on this list: Joe Montana in 1980 (#4), Tom Brady in 2001 (#14) - (see my post earlier comparing Brady's season to Cassel's here), Brett Favre in 1992 (#15), Dan Marino in 1983 (#21), and Johnny Unitas in 1956 (#22).  

Notable absentees:  Steve Young in 1986 (#154), Peyton Manning in 1998 (#208), Joe Namath in 1970 (#202)*, and Troy Aikman in 1989 (#311).  The all-time worst first year performance - Terry Bradshaw in 1970 (#337)*.

* Keep in mind that 1970 was the first year of the NFL following the merger, and 15 quarterbacks showed up in the database as having that year as their first year, although Bradshaw's 1970 season was his actual rookie season.

Are we done?  Hardly.  So after I went through this list, and still trying to objectively value Matt Cassel, I was interested in answering the following question(s):  what does the first-year tell us about a quarterback's future potential?  Do quarterbacks who have good first years, have good careers?  What about quarterbacks who have sub-par first years?  In other words, is there a correlation between year 1 performance and subsequent years?  When in doubt, it's always a good idea to take a look at the data.

Whoa!  The correlation between year 1 performance, and career performance, as measured by CMI standard deviations relative to the mean, is 0.75.  What that really means is that 57% of the variation in a quarterback's career performance can be explained by his year 1 performance.  I find that incredible.  So I looked a little deeper.  It turns out that 112 of the 336 quarterbacks only have 1 qualifying year.  And, for that group, the correlation coefficient is, well, 100%!  If I only looked at those quarterbacks whose career included at least 2 different years in which they threw enough passes to qualify, then the correlation coefficient drops to 64%.  Still quite remarkable.  

The table below breaks down the 336 first year seasons in terms of CMI standard deviations to see if we can glean any additional insight into whether a good first year translates into a good career and vice versa.


Well, the evidence is pretty clear.  Taken as two groups, players that have better than average (remember, the mean and standard deviation is relative to all quarterbacks, not just first year quarterbacks) first years tend to have longer careers and more productive careers than those that have sub-par first years.  And, breaking it down even further (in other words, just looking at the group whose first year was better than average, or the group whose first year was worse than average), you can clearly see that even within those sub-groups, the better the first year, the longer the career (although this correlation isn't nearly as strong), and the more productive the career. 

I haven't spent too much time on my blog discussing individual careers, except in the posts discussing Brett Favre and Jeff Garcia.  If I do this en masse, it would be such a powerful post - I'll basically be giving you my list of greatest passers to ever play the game, that I need to put a lot more thought into it than I have to-date.  In any case, some of the obvious names will show up on that list.  However, you'll find some exceptions as well (and I'm sure many of you will take exceptions to the list).  The point I really want to make here is that, when combining the table above, and the discussion following the list of the top 75 first years, it is quite remarkable how Troy Aikman "escaped" the trend, and turned into a great quarterback (or, more accurately, a great passer) - he truly turned out to be an exception (as for Young and Manning, even though they didn't rank well on the first year list, their first years weren't "that bad" compared to other first year seasons - as a matter of fact, Young's was "above average" compared to a typical first year).  Notice how I don't mention Bradshaw and Namath - it is because, statistically, their careers were sub-par.  When I publish my list of all-time greatest (and worst) passers, we'll get into it in more detail.

So, in trying to look at Matt Cassel's first year performance objectively, we've looked at how quarterbacks perform as their careers progress, how quarterbacks perform in their first year, who had great first years, where Cassel's season ranks on that list, and, how a first year translates into (or, is predictive of) a career.  Now perhaps, we can view Matt Cassel's 2008 year in an objective manner, and, if so inclined, you can at least evaluate the trade armed with some data.

Naturally, while I was putting this post together a question popped into my mind, and that is:  is a quarterback's second year a good indicator of their career?  What about looking at their first and second years?  Check back in a week or two to find out.

Is Matt Cassel The New Tom Brady?

Matt Cassel of the New England Patriots is having a good year.  He is 13th in the NFL in Passer Rating, after 14 games.  And, in what may seem eerily similar to one Mr. Tom Brady, he replaced Brady in the first game of the season, when Brady was lost to the season as a result of a leg injury.  Flashback to  September 2001, and the aforementioned Brady, replaced Drew Bledsoe as the quarterback of the New England Patriots.  The rest, as they say, is history.

That year, Tom Brady had a great year.  A dream year.  Replacing Bledsoe in the 2nd game of the season, he went on to lead the Patriots to an 11-5 regular season record, and on to win Super Bowl XXXVI, 20-17 over the St. Louis Rams.  Brady was selected to the Pro Bowl, and was named Super Bowl MVP.  He finished 6th in the NFL in passing, with a Passer Rating of 86.5, and he had a CMI of 0.552 (see the table below).  That year, the league average CMI was 0.497, and his CMI was 1.25 Standard Deviations above the average.  Only Rich Gannon (2.54), Kurt Warner (1.57), and Jeff Garcia (1.33) had a better year that year.  

The graph below the table shows that he started out a little slow (understandably so), and played fairly consistently throughout the year, with a few spectacular (and one not so spectacular) games thrown in.  He didn't throw an interception in his first five games, but then threw 4 in a 31-20 loss at Denver in game 6.  All in all, he didn't throw a pick in nine games, the Patriots going 7-2 in those games.

So, how is Matt Cassel doing trying to imitate Tom Brady?

As mentioned earlier, his current Passer Rating is 87.1, a shade higher than Tom Brady's 86.5.  His CMI is 0.568, also better than Brady's 0.552.  He has six games to-date where he has not thrown an interception, leading the Pats to a 5-1 record in those games.  So how does Cassel's CMI of 0.568 compare to Brady's 0.552?  This is where the comparison ends.  In 2008, the league average CMI is 0.542, so Cassel's performance is 0.49 Standard Deviations above the average.  Good, but not great.  Good for 9th out of 30 quarterbacks who have thrown enough passes to qualify.  The table below shows a game-by-game comparison of Cassel's performances compared to Brady's 2001 season.

A Slight (but necessary) Digression

Brian Burke, who does some excellent analytical work at advancednflstats.com has made this observation regarding three first-year quarterbacks - Joe Flacco of Baltimore, Matt Ryan of Atlanta, and Matt Cassel of New England.  He suggests that their performance has been improving as the 2008 season has moved on. and shows their performance, graphically illustrating it in the following manner.  He uses a measure, "Adjusted YPA" or, Adjusted Yards per Attempt, defined as [Yards -40*Interceptions + 10*Touchdowns]/Attempt, and then looks at their 4-week moving average to compare the three quarterbacks.

Well, I decided to look myself.  I have been "casually observing" that Flacco and Ryan have been having some "pretty decent" games recently (Ryan is on my Fantasy team).  So, I decided to put my newly developed CMI to the test.

Here's what it shows:

It appears to be very consistent with what Brian found.  Matt Ryan has improved the most.  CMI shows that Matt Cassel has in fact had the least improvement, if any.  However, to his credit, he has been performing at a very high level from the beginning.  Looking at the table below, you see that both Ryan and Flacco have remarkably similar attempts and completions through the first 12 weeks, with their only difference being the # of interceptions thrown.  Also, you can see that Cassel attempts more than 5 passes per game more than the other two (partially reflecting the fact that New England has no running game).

Brian has, in the past, ventured into the "creating a new passer rating" space, and, in fact, created one.  He uses a concept called "Air Yards", defined as, [Yards - Yards After Catch] to relate "passer rating" to a team's wins and losses.  See his post here.  I agree with his approach generally.  In other words, if you look at his formula, it's a lot like CMI.

QB Wins Added = [(Air Yards - Sack Yards) * 1.56 - INTs * 50.5]/Pass Attempts - 3

where, (Air Yards - Sack Yards) is substituted for Completions.  In other words, he also takes out Touchdowns thrown and Yards per Attempt.  Brian is the first person I know of that does not include touchdowns as a part of a passer rating calculation, and he is to be commended for that.  My only real issue with Brian's calculation is that YAC, or, Yards After Catch, is not a readily available statistic, and hence, "Air Yards" is not easily calculated.

I am still developing CMI, and the next (enhanced) version will incorporate sacks.  Given that one of my goals is also to look back in time and be able to compare QBs over time, I will only be able to that back to 1969.

As I mentioned at the top, Brian is a very smart guy, and has done some very good work on a whole host of topics relating to the NFL, with particular attention to statistical rigor.  I will be referring to his site quite a bit in future postings.