2009 NFL Season - Passer Ratings

We now have data for the entire 2009 season, and I can calculate the best and worst performers for the season, using my standard measure, CMI - or more appropriately, Standard Deviations from mean CMI.  CMI is easily calculated, as it is simply [pass completion percentage] - 3 * [interception percentage].  I can calculate that at any point during the season.  In order to calculate the standard deviations from the mean CMI, I like to wait until the season is over (actually, even this can be calculated at any point during the season, but it's a lot simpler for me to just do it at the end of the season - perhaps next year, I can do this after every week during the regular NFL season).   For those of you familiar with statistics, the measure of standard deviation from the mean is also called a z-score (or z-value).  And, furthermore, z-score can be very easily converted to a percentile rank (assuming that the population is normally distributed.  In a later post, I will show that z scores for CMTI or NFL passer rating over the years is distributed normally).  The beautiful thing about a percentile rank is that it is always a number between 0 and 100.  What's not to like about that!

A quick couple of notes here regarding CMI, and many of the calculations that I will be using from here on out. While I will go through the exercise of creating arduous, complex and sometimes arbitrary (and even unnecessary) calculations, I will strive for simplicity whenever possible when I present my ideas, as I believe they are more likely to be accepted.   The use of percentile ranks is an example of this.

Last year, when I first developed CMI, and especially as I looked back in time, I had to figure out how many players to use each year to determine the number of qualifiers.  I went through some elaborate mechanisms to calculate the number of passes attempted during the season (per team per game), and some qualifying standard.  As you are aware, the NFL uses 14 passes per game as the standard.  And yes, while it is simple to use, I am not a big fan of that standard since the average # of passes attempted per game has changed (increased) over time, as you can see from the table below.

The standard today should be closer to 16.

Last year, I arbitrarily assigned a factor close to 50% (with adjustments for the earlier years) to the total # of passes attempted by each team per game to determine the # of qualifiers.  I realized this year, that if I simply used the # of teams in the league as the cutoff for determining the # of qualifiers, I get fairly close to my original method.  And because it is simple, easy to explain, does a better job (in my opinion, anyway) than the current NFL standard, I will use that as the cutoff.  The table below shows the # of qualifiers by season using the NFL standard, my original standard, and my current standard.

As you can see, the # of qualifiers, regardless of which method you select is fairly close in each year, with the exceptions being the earliest years, where the NFL method simply excludes too many passers (in my opinion anyway).  Also note that the the AAFC is excluded by the NFL altogether, while the AFL and NFL have been combined in the years 1960-1969 by the NFL.  This doesn't make sense to me, since each league played a different # of games (and had a different # of teams).  Regardless, if you combine the AFL and NFL results from my method, you get close to the #s used by the NFL.  While I think that my method works, I also recognize that this standard may not be the most appropriate for future NFL seasons.

OK, back to the passer ratings.

Because I will discuss other passer rating systems/calculations that are related to CMI in later posts, I will take this moment to re-classify CMI as CMTI (acronym stands for Completions Minus Three times Interceptions).  The definition itself hasn't changed - just the acronym.

The table below shows each of the qualifying quarterbacks with their 2009 season's statistics, their NFL passer rating, the percentile rank of their 2009 NFL passer rating (in other words, the converted z-scores), their 2009 rank in terms of NFL passer rating, their 2008 NFL passer rating, the percentile rank of their 2008 NFL passer rating, their 2008 NFL passer rating rank, their 2009 CMTI, 2009 CMTI percentile rank, their 2008 CMTI and 2008 CMTI percentile rank, and their 2008 CMTI ranking, and finally, their average 2-year CMTI percentile rank (again a # between 0 and 100), together with that ranking.

I don't think there's any question as to who the elite quarterbacks are.  Also notice the struggles of first-year quarterbacks in each year.

Better Than Jay Cutler

A lot of noise this past month about Jay Cutler. First, the Denver Broncos' new head coach, former New England Patriots offensive coordinator Josh McDaniels, apparently tried to work a trade to get Matt Cassel from the New England Patriots. This apparently didn't it well with the Pro-Bowl quarterback. After a lot of posturing from both sides, Cutler was eventually traded to the Chicago Bears for Kyle Orton and draft picks. This post isn't about who was right or wrong in the fracas, or which team got the better end of the deal in the trade. This post is about answering the following two questions:

1) How good is Jay Cutler?

2) Are there quarterbacks with similar experience that are better than Jay Cutler, and, if so, who are they?

The table below ranks each of the 30 quarterbacks that threw enough passes last year to qualify to be ranked. The ranking is based on CMI, and not the NFL's quarterback rating system. In numerous posts prior to this, I have discussed why CMI is a better measure of a quarterback's passing performance than the NFL passer rating system.

Just looking at last year, the year that Jay Cutler got named to his first Pro-Bowl, he was, by this measure, a below-average quarterback. His CMI of 0.536 was (0.14) standard deviations below the mean. It should be noted that even though he was below average relative to all quarterbacks in 2008, it wasn't unexpected. Previously, I discussed the notion of performance improving with experience. So, if you were to look only at quarterbacks and their second year performance, their average is (0.21) standard deviations relative to the mean. Relative to that measure, he actually did better than the typical second year quarterback.

Let's take a closer look at the table above. The last 2 columns indicate the # of years the quarterback has played since he first qualified, and the total # of years he has qualified. So, Cutler has played 2 seasons where he threw enough passes to qualify, and this is second consecutive year of being qualified. There are six other quarterbacks in this table who have similar experience.

They are (rank in parenthesis):

Jason Campbell - WAS (5)

Matt Schaub - HOU (7)

Trent Edwards - BUF (8)

David Garrard - JAX (12)

Jay Cutler - DEN (16)

Derek Anderson - CLE (30)

Of the six quarterbacks who were in their second year (Garrard has the most experience of all the quarterbacks, even though he has only qualified in two years), Cutler only outperformed Derek Anderson of Cleveland in 2008. Ok, so 1 year does not make a quarterback. Let's take a look at these six quarterbacks and see how they've done in each of their first two (qualifying) years.

According to this table, both Garrard and Schaub have outperformed Cutler in each of 2007 and 2008. And, arguably, over the two-year period, so did Campbell. The only quarterback in this class that has clearly underperformed the rest of the group is Derek Anderson. He will not last long in the NFL.

Admittedly, it's difficult to judge a quarterback after his first two years. But, so far at least, there appear to be several quarterbacks that have outperformed Cutler. It will be interesting to watch this group over the next several years, and especially Cutler, as he will be with a new team.

A Few Weeks Off

I took a few weeks off from posting.  It was necessary for several reasons.  

First, I do have a day job as the Chief Financial Officer of InsWeb Corporation, a publicly traded company (NASDAQ CM: INSW).  The months of March and April, I get particularly absorbed in my work as we have to officially close the books for the prior fiscal year (in this case the year ending December 31, 2008), and we have to get prepared for closing the books for the first quarter of 2009.  This is followed by a meeting with our Board of Directors.  It is an extremely busy time, and a very important time for our business.  

Second, posting a blog (the way I do it), takes a lot of time, as I spend considerable time before posting doing a lot of research, as I like to support my arguments with facts and analyses.  

Third, I got distracted by a couple of things.  The "event" known as March Madness - the NCAA basketball tournament, and the 2009 Major League Baseball season.  Well, not the baseball season per se, but the fantasy baseball leagues that I participate in.  Preparing for these takes up a lot of time.

Fourth, I needed a break.

Well, I'm back at it.  I have two immediate topics that I'd like to cover.  

The first is the trade of Jay Cutler from the Denver Broncos to the Chicago Bears.  Quick, name three quarterbacks who, like Cutler, finished their second year in the NFL (technically, 2008 was Cutler's third season, but the way I count it, it was his second season throwing enough passes to qualify for a rating) last year that are arguably better than Jay Cutler.  Didn't think you could do it.  I'll have the answers in my next post.  The answers might surprise you.

The second is the NFL draft.  I have a lot of observations about the draft.  I won't be discussing whether Matthew Stafford will be a better NFL quarterback than Mark Sanchez.  Why?  Because I don't know.  And neither do you.  And nor does anyone else.  Only time will tell.  And even then, we may never know.

The Best and Worst of 2008

This is a simple post.  Really simple.  The list should pretty much speak for itself.  The best and worst of 2008.  I use the same criteria I've been using the past few posts.  CMI, and, more specifically, standard deviation of a player's CMI relative to the mean.  30 players attempted enough passes in 2008 to make the list.  

The Best 3:

3.  Jeff Garcia, Tampa Bay Buccaneers - Why no team has picked him up is a complete mystery to me - the man can still play, even though his durability is certainly a question.  Completed nearly 65% of his passes, and less than 1.6% of his passes were intercepted.

2.  Peyton Manning, Indianapolis Colts - Do we even need to say anything? Coming off off-season knee surgery, Mr. Consistency now holds the highest current active streak of games started consecutively - 176 games. He completed almost 67% of his passes.

1.  Chad Pennington, Miami Dolphins - Let go by the Jets so that they could welcome Brett Favre (that worked out well for the Jets, didn't it?) and subsequently picked up by the Dolphins. Had a terrific season, completing more than 67% of his passes, and less than 1.5% of his passes were intercepted.

In previous posts on my blog, I have spent time discussing each of the above 3 players.

The Worst 3:

28.  Tyler Thigpen, Kansas City Chiefs - Won't show up at the bottom of the list in 2009.  Why?  Matt Cassel (#10 in 2008).  The Chiefs made a deal with the New England Patriots to get Cassel (and Mike Vrabel) for a 2nd round pick in the upcoming draft.  In 2008, Thigpen, replacing Damon Huard, completed fewer than 55% of his passes.

29.  Gus Frerotte, Minnesota Vikings - Replaced an ineffective Tarvaris Jackson in the 3rd game of the season, started 11 games, and then was himself replaced by Jackson against the Lions.  Completed less than 60% of his passes, and at almost 5.0%, was one of only three quarterbacks whose interception percentage exceeded 3.0% - the other two being Ben Roethlisberger of the Steelers (3.2%), and Brett Favre of the Jets (4.2%).

30.  Derek Anderson, Cleveland Browns - Barely completed 50% of his passes, then got hurt.  Was replaced by Brady Quinn.  Is it Quinn's time in 2009?

Here's the entire list.

My next post - a pre-season ranking for 2009

First Year Quarterback Performance - An Evaluation Method

This past weekend, the New England Patriots traded Matt Cassel to the Kansas City Chiefs.  In fact, they traded linebacker Mike Vrabel and Cassel for a 2nd round pick (pick #34) in the upcoming April draft (see here for the details).  Depending on what your news source is, this was labeled as either a good deal for Kansas City, or a bad deal for New England.  The experts discussing this issue don't seem to support their arguments by any objective measures - they just opine.  I am not going to jump into the debate here.  I will offer this however - we will probably not know who will end up on the better end of this deal until many years from now, and even then, we may not know.  

What about the objective evidence?

Matt Cassel is not the only quarterback in 2008 that had a good year as a first-year quarterback.  There was Aaron Rodgers of the Packers, Shaun Hill of the 49'ers, and Matt Ryan of the Falcons.  There were four other quarterbacks who threw enough passes this year to make the list of first-year quarterbacks - Ryan Fitzpatrick of the Bengals, Joe Flacco of the Ravens, JaMarcus Russell of the Raiders and Tyler Thigpen of the Chiefs.  

Notice here that I didn't use the term rookie quarterback, but instead used the term first year.  The way I look at a quarterback, is that I only consider a quarterback who has thrown "enough" passes to qualify in a given year.  The # of passes has varied by year (as opposed to the fixed standard used by NFL.com).  For the 69-year period from 1940-2008, my database contains 1,451 passing seasons, 336 of which were by a quarterback qualifying for the first time.

How did the aforementioned 8 quarterbacks do?  

I use a statistical measure I have previously developed and discussed, called CMI (Completions Minus Interceptions, calculated as Completions/Attempts - 3*Interceptions/Attempts) instead of the NFL Passer Rating system (using the NFL passer rating system gets you essentially the same answers), and then use the standard deviation from the mean as the measuring stick (for the pool of qualifying passers, I calculate the the mean and standard deviations for each year, and relate a particular performance in that year to the mean using that year's standard deviation).  Using the mean and standard deviation this way not only allows me to compare these 8 quarterbacks relative to each other, but also relative to all quarterbacks this year, or any other year.

In a post not too long ago, I showed how, over the past 69 years from 1940-2008, when the standard deviation is used as the measuring stick, the accumulated data form a near-perfect theoretical standard normal distribution.  And I also showed how, this is true regardless of whether one used the NFL passer rating system or CMI as the statistical base.

I can now show you how these first year quarterbacks did compare to the rest of the QBs in 2008, using this measure.  But this picture may not tell us much, if anything.  Here's the graphical illustration:

Perhaps the only thing that may be apparent from this is that first year quarterbacks don't do nearly as well as quarterbacks with more experience.  Even drawing that conclusion from this is a little iffy.  However, it is clear (at least by this measure) that Cassel had the best year of the 8 first year QBs.  Rodgers was a close second.  JaMarcus Russell and Tyler Thigpen had disastrous first years.  In an earlier post regarding Jeff Garcia, I had suggested that Kansas City might be a good place for Garcia.  Scott Pioli, the new Chiefs' GM apparently was thinking the same thing - upgrade at quarterback, and he did - with Cassel.

Going back to the standard normal curve, let's look at just the first year performances.  Recall that, when all 1,451 seasons are aggregated in a histogram, it looks very much like a standard normal curve.

This doesn't quite look "normal", does it?  The first year performances appear skewed to the left, or, compared to all quarterback performances, appear to under-perform.  Since the total database is normally distributed, then performances in years 2 and beyond must obviously be skewed to the right.  This prompts the question: does experience improve performance, and, in particular, how and when?

One way to look at this is to take the entire database, and look at the average standard deviation from the mean for each additional year a player qualified.  Here's what that graph looks like.

Wow!  I didn't realize that it takes about 5 qualifying years to turn into an average quarterback.  And that's just for those playing that long!  Even though this graph goes out to 18 years, there is hardly any data beyond the 15th year (there are only 8 quarterbacks who have had qualifying seasons 16 different years).  According to this illustration, basically, first year quarterbacks, as a group, struggle mightily.  They then gradually improve each year until about year nine.  Quarterbacks, on average, must still endure sub-par years in years 2-4, and for those that survive, their reward is about four more years of improving performance, assuming they stay healthy for that long.

Note: This analysis - the method of breaking down a quarterback's expected career in terms of passing performance as it relates to experience and how it is expected to change over time, is the first I've seen published publicly.  But I digress.

Ok, so we've taken a look at these 8 QBs relative to all quarterbacks in 2008, and we've taken a look at first year quarterbacks in general.  So the next step is to take a look at the best first year seasons.  In my database, there are 336 first year seasons in all (in other words, in the 69-year period from 1940-2008, 336 different quarterbacks threw enough passes in a season to qualify at least once).  The table below shows the 75 best.

Cassel (#48) and Rodgers (#60) are the only 2 of the 8 in 2008 that crack the top 75 all-time.  On the other end of the scale, we get JaMarcus Russell (#264) and Tyler Thigpen (#290).

When Matt Ryan (#128) got off to a solid start, and the Falcons surprised many by getting into the playoffs, there were many debates as to whether his season was one of the best ever by a first year player.  Not according to this measure.  Often, during those discussions, Kurt Warner's first year was brought up.  In looking at the table above, well, he did have an outstanding first year in 1999.  It ranks #6 all-time in terms of first year seasons.  The best ever - Roger Staubach's 1971 season (Staubach was a rookie in 1966, and played in both 1969 and 1970.  However, he didn't attempt enough passes in either year to qualify).  

Notable quarterbacks high on this list: Joe Montana in 1980 (#4), Tom Brady in 2001 (#14) - (see my post earlier comparing Brady's season to Cassel's here), Brett Favre in 1992 (#15), Dan Marino in 1983 (#21), and Johnny Unitas in 1956 (#22).  

Notable absentees:  Steve Young in 1986 (#154), Peyton Manning in 1998 (#208), Joe Namath in 1970 (#202)*, and Troy Aikman in 1989 (#311).  The all-time worst first year performance - Terry Bradshaw in 1970 (#337)*.

* Keep in mind that 1970 was the first year of the NFL following the merger, and 15 quarterbacks showed up in the database as having that year as their first year, although Bradshaw's 1970 season was his actual rookie season.

Are we done?  Hardly.  So after I went through this list, and still trying to objectively value Matt Cassel, I was interested in answering the following question(s):  what does the first-year tell us about a quarterback's future potential?  Do quarterbacks who have good first years, have good careers?  What about quarterbacks who have sub-par first years?  In other words, is there a correlation between year 1 performance and subsequent years?  When in doubt, it's always a good idea to take a look at the data.

Whoa!  The correlation between year 1 performance, and career performance, as measured by CMI standard deviations relative to the mean, is 0.75.  What that really means is that 57% of the variation in a quarterback's career performance can be explained by his year 1 performance.  I find that incredible.  So I looked a little deeper.  It turns out that 112 of the 336 quarterbacks only have 1 qualifying year.  And, for that group, the correlation coefficient is, well, 100%!  If I only looked at those quarterbacks whose career included at least 2 different years in which they threw enough passes to qualify, then the correlation coefficient drops to 64%.  Still quite remarkable.  

The table below breaks down the 336 first year seasons in terms of CMI standard deviations to see if we can glean any additional insight into whether a good first year translates into a good career and vice versa.


Well, the evidence is pretty clear.  Taken as two groups, players that have better than average (remember, the mean and standard deviation is relative to all quarterbacks, not just first year quarterbacks) first years tend to have longer careers and more productive careers than those that have sub-par first years.  And, breaking it down even further (in other words, just looking at the group whose first year was better than average, or the group whose first year was worse than average), you can clearly see that even within those sub-groups, the better the first year, the longer the career (although this correlation isn't nearly as strong), and the more productive the career. 

I haven't spent too much time on my blog discussing individual careers, except in the posts discussing Brett Favre and Jeff Garcia.  If I do this en masse, it would be such a powerful post - I'll basically be giving you my list of greatest passers to ever play the game, that I need to put a lot more thought into it than I have to-date.  In any case, some of the obvious names will show up on that list.  However, you'll find some exceptions as well (and I'm sure many of you will take exceptions to the list).  The point I really want to make here is that, when combining the table above, and the discussion following the list of the top 75 first years, it is quite remarkable how Troy Aikman "escaped" the trend, and turned into a great quarterback (or, more accurately, a great passer) - he truly turned out to be an exception (as for Young and Manning, even though they didn't rank well on the first year list, their first years weren't "that bad" compared to other first year seasons - as a matter of fact, Young's was "above average" compared to a typical first year).  Notice how I don't mention Bradshaw and Namath - it is because, statistically, their careers were sub-par.  When I publish my list of all-time greatest (and worst) passers, we'll get into it in more detail.

So, in trying to look at Matt Cassel's first year performance objectively, we've looked at how quarterbacks perform as their careers progress, how quarterbacks perform in their first year, who had great first years, where Cassel's season ranks on that list, and, how a first year translates into (or, is predictive of) a career.  Now perhaps, we can view Matt Cassel's 2008 year in an objective manner, and, if so inclined, you can at least evaluate the trade armed with some data.

Naturally, while I was putting this post together a question popped into my mind, and that is:  is a quarterback's second year a good indicator of their career?  What about looking at their first and second years?  Check back in a week or two to find out.

A Brief History of the NFL Passer Rating System

Philip Rivers of the San Diego Chargers had a phenomenal season in 2008.  Or did he?  According to the NFL's passer rating system, Rivers' season, where he completed 312 passes out of 478 attempted, threw for 4,009 yards, and 34 touchdowns against 11 interceptions resulted in a passer rating of 105.5.  That's great, right?  Before we answer that question, we'll need to answer a few more questions.

How do the statistics 312 for 478, with 4,009 yards, 34 touchdowns and 11 interceptions translate to a rating of 105.5?

What does having a rating of 105.5 mean?

Where does Rivers' season rank all-time?

How has the average NFL passer rating changed over time, and how does that affect how we look at Rivers' season?

The NFL began keeping statistics in 1932.  The NFL has used various methods to determine the "best" passer in the years since.  According to the Pro Football Hall of Fame, the following is a chronology and description of the different methods used:

1932-1937: Total yards passing

1938-1940: Percentage of completions

1941-1948: Inverse ranking system of the following categories: completions, percentage of completions, total yards, total TD passes, number of interceptions, and percentage of interceptions

1949: The same formula used from 1941-1948 except the number of interceptions were dropped from the equation

1950-1959: Average yards gained per pass with a minimum of 100 attempts needed to qualify

1960-1961: Inverse ranking system based on six categories: total completions, total yards, total TD passes, percentage of completions, percentage of interceptions, average gain per attempt with the principle established of at least 10 attempts per game to qualify

1962-1971: Inverse ranking system based on four categories: percentage of completions, total touchdown passes, percentage of interceptions, average gain per attempt

1972: Same system used from 1962-1971 except that the percentage of touchdown passes was substituted for total touchdown passes

1973 to present:  See below.

Current NFL Passer Rating System:

The passer rating system used by the NFL has been in place since 1973.  In 1971, after the merger, then-commissioner Pete Rozelle wanted to implement a standardized set of statistics, including a standard measure of passing performance.  As the description on the Hall of Fame site suggests, this is a measure of a quarterback's passing effectiveness, not a measure of how good a quarterback is.  Rozelle called upon Don Smith, then an executive with the Hall of Fame, to work with the league's official statistician, the Elias Sports Bureau, to develop a new standard.  

One of the problems with the standard that existed at the time, was that you didn't know where you stood until all the teams' quarterbacks had finished playing, as it was a relative ranking system.  In addition, there wasn't a convenient way to compare a performance in a given year to that of another.

Don liked the use of the combination of statistics - in other words, the completion percentage, the yardage per attempt, the touchdown percentage and interception percentage.  He had to figure out how to use all four in some manner that would make sense.

Working with the Elias Sports Bureau, he studied passing statistics for each of the four categories from the decade before - the sixties.  After much study and thought, he devised a system whereby, for each of the four statistical measures, he would convert that particular measure to a "score" between 0 and 2.  A score of 0 would indicate poor performance, a score of 1.00 would indicate "average" performance, and a score of 2.00 would indicate "superior" performance.  With truly exceptional performance, it was possible to exceed a score of 2.00.  He arbitrarily chose the maximum score to be 2.375.  

He decided that an "average" performance across all four categories, that is, a score of 1.00 for each of the four measures, should be a rating of 66.7.  Under this new rating system, it would be possible for a rating to exceed 100.0, but, he theorized that those instances would be rare.

(Note:  I took a lot of this history from an article written by Don Steinberg, published in Slate magazine in 2001 - see here for the full article)

In summary then, this is how we end up with the current NFL passer rating system:  we take four components of passing, convert each component, using league averages from the 1960's, to a score with a minimum of 0.000 and a maximum of 2.375, combine these scores by adding them, and convert to a rating system that has 66.7 as the "average".

Here's the math that does this:

Q = [ ( J + K + L + M ) * 100 ] / 6

where,

Q = Passer Rating

J = max [ min ( C, 2.375 ), 0 ]

K = max [ min ( Y, 2.375 ), 0 ]

L = max [ min ( T, 2.375 ), 0 ]

M = max [ min ( I, 2.375 ), 0 ]

and where,

C = [ ( Completions / Attempts ) * 100 - k1 ] / 20

Y = [ ( Yards / Attempts ) - k2 ] / 4

T = [ ( Touchdowns / Attempts ) * k3 ] * 20

I = 2.375 - [ ( Interceptions / Attempts ) * k4 ] * 25

and where,

k1 = 30

k2 = 3

k3 = 1

k4 = 1

In the first formula, you can easily see where if J = K = L = M = 1.000, the passer rating formula would yield Q = 66.7.  In order to get J = K = L = M = 1.000, certain "transformations" were needed to each of the passing statistics to convert the averages to a score of 1.000.  The transformations for each are shown as C, Y, T, and I, and, more specifically, k1, k2, k3 and k4.  The figures for k1-k4 were derived using league statistics from the 1960's, and to make it somewhat easier, rounded.  If you looked at data from 1960-1969, you get the following actual values for k1-k4:

k1 = 31.70

k2 = 3.24

k3 = 0.96

k4 = 0.99

The table below shows, for each NFL season from 1940-2008, how the averages, or, more precisely, the "scores" for those averages have changed from year to year (for a graphical illustration see my previous post on the subject here).  For example, you can see that the average score for "J", in 2008 was 1.550, reflecting the fact that the "C" component, or, completions per attempt, for the average NFL quarterback was 61.0%.  Compare this to the average score for "J" in 1968 of 1.079, reflecting the "C" component, or completions per attempt of that time of 51.6%.  This is what happens when the transformations do not change over time, even when the actual game itself has undergone many transformations.  If one wanted to keep everything in balance from year to year in the passer rating formula, in other words, to keep J = K = L = M = 1.000 for every year, then one would have to change the values of k1 - k4 every year.  I have done that in the table below.

You can use these different k values every year to "adjust" the passer rating formula, so that you can make meaningful comparisons from year to year.  If you didn't do that, and simply compared a quarterback rating from one year to the next, you wouldn't get an appropriate comparison, for the simple reason that the four components would be "out of balance".  

While it was never Don Smith's intention to have a relative measure of performance - he wanted performance as measured relative to a fixed standard, that "fixed" standard has changed, and will continue to change over time.  One way to minimize the effect of these changes is to look at a given performance in a given year to that season's average, using the standard deviation as a measuring stick.  The other way, is to simply "adjust" the standard to reflect the averages for that year.  In a previous post, I discussed the former method.  In this, I am obviously discussing the latter.

This discussion is only to put Rivers' 2008 season in perspective.  It is not intended as passing judgment on the NFL passer rating system.  I have, in previous posts (see here and here) discussed why I do not like the system.  In a post in the future, I'll elaborate further on why I think the NFL's passer rating system should be revised.

The table below shows the 75 greatest seasons in the NFL, both in terms of the current NFL passer rating system, and using an "adjusted" passer rating system, going through the transformations as I have described above.  You can see that, using the current system, that Rivers' 2008 ranks as 13th best all-time.  Using an "adjusted" passer rating system, however, results in a ranking of 60th best all-time.  If we used the previously mentioned standard deviations from mean measurement, it would rank as the 16th best all-time.

We still have some unfinished business:

How does Rivers' stats translate to a rating of 105.5?

He completed 312 of 478 passes, or, a completion percentage of 65.3% - using k1 = 30, and Comp/Att = 0.653 in the formulas above, yields a J value of 1.764

He threw for 4,009 yards, or, yards per attempt of 8.4 - using k2 = 2, and Yards/Att of 8.4, we get a value for K of 1.347

He threw 34 TDs, or, TD/Att of 7.1% - using k3 = 1, and TD/Att = 0.071, we get a value for L of 1.423

He threw 11 INTs, or, INT/Att of 2.3% - using k4 = 1, and INT/Att - 0.023, we get a value for M of 1.800

Combining all, we get

Q = ( J + K + L + M ) * 100 / 6, or

Q = ( 1.764 + 1.347 + 1.423 + 1.800 ) * 100 / 6 = 105.5

There is no doubt that, whether we use the current NFL passer rating system, an "adjusted" passer rating system with different transformations for different years, or, the current system, but adjusted by looking at standard deviations from the mean, that Rivers' 2008 season was quite remarkable.  

Whether the passer rating system is an accurate reflection of a passer's ability is a completely different question altogether.  We shall, over time, address this question.

Anybody Looking for a Good QB?

Available:  Experienced quarterback with an accurate arm with ability to make immediate contributions to team; Ideally, would like to start, but not a requirement; may be perfect as back-up; durability maybe an issue.

I am referring of course to Jeff Garcia, released today by the Tampa Bay Buccaneers.  I am personally a big fan of Garcia, because I believe he is underrated, and his achievements on the field have gone relatively unnoticed.

Let's take a look.

He began his NFL career in San Francisco in 1999, replacing not one, but two back-to-back Hall of Famers, and arguably the two greatest passers of all-time, Joe Montana and Steve Young.  There was no way Jeff could live up to those lofty expectations.  However, he did perform very well, as the table and chart below shows.  While he may not have performed at Joe Montana and Steve Young levels, his years in San Francisco were very productive.  After a sub-par year in 2003, he bounced around for three years, first as a starter in Cleveland (2004), then as a back-up in Detroit (2005) and Philly (2006).  He finished the 2006 season as the starter in Philadelphia for the last 6 games after Donovan McNabb got injured.  He showed that he could still play, and Tampa Bay, in need of a quarterback following an injury to Chris Simms, picked him up.  He did well above average each of 2007 and 2008, as you can see from the table and charts (he doesn't show up in the charts in the years 2004-2006 since he didn't attempt enough passes to qualify).  

Now, Tampa Bay, deciding to turn to the youth movement, have invested in Luke McCown, and, have deemed Garcia expendable.  

Garcia has been a very accurate as a passer throughout his career.  In a previous post, I discussed interceptions at length.  His 2007 season ranks 18th all-time (see Exhibit 3 in this post) in interceptions per attempt, in terms of standard deviations from the mean (his 2008 season ranks 118th, out of the 1,451 seasons in the database since 1940).  As a career passer, he ranks 12th all-time (see Exhibit 6 in this post).  By not throwing interceptions, he will help your team win.

I believe that Garcia has a couple of good years still left in him, if not as a starter, as a back-up.  His durability has been an issue the past two seasons, and hence, might be a liability as the starter.  

So who might be worthwhile candidates?

The following charts show, for each of 2007 and 2008, how a team's passing game stacked up in terms of standard deviations from the mean.  It's no coincidence that teams that were above average won more games than they lost, and teams that were below average lost more games than they won.  The charts show how the teams did both in terms of NFL passer rating and CMI.  Note that Garcia averaged 0.53 standard deviations above average in terms of NFL passer rating for the past two seasons, and averaged 1.23 standard deviations above average in terms of CMI.

To see which teams should be looking to get Garcia, let's start at the bottom, and move our way up.

Cleveland Browns - They have two young quarterbacks, Derek Anderson and Brady Quinn, both coming off season-ending injuries, that will compete for the starting job.  Not a candidate.  Look for another miserable year in Cleveland.

St Louis Rams - Marc Bulger is the starter, and for now at least, another soon-to-be 39 year-old - Trent Green, is the back-up.  Not a candidate.  By the way, what's happened to Bulger?  Will 2009 be more like his first few years, or his last two?  I suspect the latter.  Another long year in St Louis.

Oakland Raiders - Paying a lot of money for JaMarcus Russell.  They've also got Andrew Walter, Marques Tuiasosopo, and they just picked up Bruce Gradkowski off waivers.  I haven't a clue what the Raiders are doing, and I don't think the Raiders do either.  Not a candidate.  Perhaps we'll start paying attention when Al Davis passes away.

San Francisco Forty Niners - They're not sure who their starting quarterback is.  Shaun Hill?  J.T. O'Sullivan?  Alex Smith?  The niners maybe a candidate.  Garcia played here, and is originally from close-by Gilroy.

Detroit Lions - Daunte Culpepper is the starter.  Or is it Dan Orlovsky?  Didn't seem to matter in 2008, as they went 0-16.  If I was GM, I'd replace Culpepper with Garcia, and draft a quarterback.  But that's just me.  Garcia may not want to go here, but that's a different matter altogeher.

Chicago Bears - Kyle Orton is the starter, and Rex Grossman is the back-up.  Neither of whom are very good.  I would think that Garcia would be an excellent choice to be a back-up here.

Kansas City Chiefs - Tyler Thigpen is the starter.  Previous starter Damon Huard is the back-up.  Garcia could be a potential back-up here.

Cincinnati Bengals - Ryan Fitzpatrick finished the year as the starter, having replaced the injured Carson Palmer.  Not a candidate.

Of the remaining teams, perhaps the Minnesota Vikings might be interested.  Gus Frerotte is simply not a good option.  Maybe they'll lure Brett Favre out of retirement!  The NY Jets might be an option for Garcia as well.  Most likely, Kellen Clemens will be the starter, but Garcia could be a viable back-up here.  

So these are teams that should be interested in Garcia.  We'll wait and see where he actually ends up.  

The Importance of Interceptions (or lack thereof)

I have spent quite a bit of time lately talking about interceptions.  In case you have any doubt that an interception can make a difference you got that answer on Super Bowl Sunday.  In probably one of the greatest plays in Super Bowl history (until the catch by Santonio Holmes with 35 seconds left that gave the Steelers a come-from behind victory), with Arizona on the Pittsburgh 1 yard line, first and goal, and 18 seconds left in the first half, the Steelers' James Harrison picked off Arizona's Kurt Warner and returned it all the way for a touchdown as time expired.  This was, in effect, a 13-point play, as Arizona's expected points at the Pittsburgh 1 was about 6 points.  Brian Burke, who I've highlighted before, has an excellent post on the subject on his site at advancednflstats.com.   

Take a look at the graph below.  The blue line (on the left scale) shows the league average QB passer rating (for those QBs who thew enough passes during the season to qualify) by year since 1940.  As you know, the NFL's QB passer rating formula has four components - completion percentage, yards per attempt, touchdown percentage, and interception percentage (see my previous posts on the subject here and here).  When it was designed in 1973, the formula used the 1972 season as a "base", and hence created adjustments to each component, such that the average would be a score of 1.00 for each component, resulting in a passer rating for 66.7 for a quarterback who had average statistics in each of the four categories.  The actual calculations for each of the four components in 1972 yielded the following four figures - 1.085, 0.954, 0.897, and 1.043, respectively, which in turn yielded the average quarterback passer rating of 66.3 (the figure for the qualified leaders turns out only slightly higher - 67.9).  Back then, each of the four components were essentially balanced.

Today, it's a different story.  If the system was balanced, then we would expect the interceptions component to make up about 25% of the quarterback passer rating score.  Looking at the graph again, and this time looking at the red line (on the right scale) shows how much the interceptions component influences the league's quarterback rating system.  It hasn't been below 30% since 1983, and the last time it was "around" 25% was actually 1971.  The point is that the NFL passer rating system is not what it used to be, and interceptions are the leading weight in today's NFL passer rating system.

Let's take a look at another example.  Tom Brady's record-breaking 2007 season.  That year, Brady completed 398 of 578 passes, for 4,806 yards, and threw for 50 touchdowns while only throwing 8 interceptions.  His passer rating that year - 117.2, is second only to Peyton Manning's all-time best 121.1 in 2004.  Most people will remember the 50 touchdowns that Brady threw.  Impressive as it was, it wasn't that impressive (I'll have a post later on this subject).  Most people will not remember that he only threw 8 interceptions in 578 attempts.  That ranks as 14th best all-time in terms of single-season interception percentage (see Exhibit 2), and, it accounted for 29% of his quarterback passer rating that year (his touchdowns accounted for 25%).

There are 7 exhibits on interceptions attached to this post.   Note that for all exhibits, I only considered those passers in any given year that threw enough passes to qualify - in other words, if a quarterback threw 20 passes during a season, that would not be enough to qualify him.  On the nfl.com site, the minimum qualification standard is 14 passes per game (so, for the 2008 season, the standard would be 14 * 16 = 224 passes).  The NFL uses this standard EVERY year.  

I think this universal standard is inappropriate.  Using this method for example, only three passers qualified in 1940.  The 14 passes comes from the fact that during the 50's, 60's and 70's, the average number of passes attempted by a team during a game was about 28.  Of course, we all know that this has changed over time.  The average number of passes attempted in a game by a team was 32.3 in 2008.  It has been below 30.0 once since 1979.  

The standards that I use varies by year, and reflects the average # of passes attempted during these years.  Here are the standards:

1940-1946 - 6.5

1947-1960 - 11.0

1961-1969 - 14.0

1970-1977 - 12.0

1978-1994 - 15.0

1995-2008 - 16.0

Using these standards, I get 1,451 quarterbacks in my database, with a low of 9 quarterbacks qualifying in each of 1941 and 1943, and a high of 32 quarterbacks qualifying in each of 1999 and 2005.  For 2008, I had 30 quarterbacks in my qualified database, whereas the NFL.com has 32.  So, not a big difference in recent years.  I just think that applying a universal standard across all these years is silly, especially when the game has changed so much.

In any case, now that we got the some of the technical stuff out of the way, here are the 7 exhibits:

Exhibit 1 - Chronological list of NFL leader (lowest) in interception percentage

Exhibit 2 - Best seasons in terms of interception percentage

Exhibit 3 - Best seasons in terms of standard deviations from the mean

Exhibit 4 - Worst seasons in terms of standard deviations from the mean

Exhibit 5 - Best (lowest) career interception rate (minimum of 1,000 passes attempted)

Exhibit 6 - Best career interception rate relative to league average

Exhibit 7 - Worst (highest) career interception rate relative to league average

Exhibit 1 - Chronological list:

Nothing spectacular here, it's simply each year's best.  I observe a few things: 

Clearly, the average interception rate has been decreasing every decade.  

The best quarterbacks in a given year seem to be around 1.8 standard deviations better than the average.

Slinging Sammy Baugh led the league 4 out of 6 years during the period from 1942 to 1947 - the only quarterback to have led the league on 4 separate occasions.  

6 quarterbacks - Bart Starr, Bobby Thomason, Charlie Conerly, Ken Anderson, Ken O'Brien, and Roger Staubach have led the league on 3 different occasions.

5 quarterbacks led the league in consecutive seasons - Sammy Baugh, Bobby Thomason, Milt Plum, Ken Anderson and Ken O'Brien.

Only 2 quarterbacks in history, Steve DeBerg in 1990 (0.90%), and David Garrard in 2007 (0.92%) completed a season where less than 1 percentage of their attempted passes were intercepted (this is not entirely evident by looking at Exhibit 1, but can be confirmed by Exhibit 2).  What's most unusual about DeBerg's performance that year was the fact that of the four interceptions he threw during the season, three of them were in one game!  In other words, he threw 1 INT the rest of the season.  Let's take a look at Exhibit 2.

Exhibit 2 - Best seasons in terms of interception percentage:

So indeed, only 2 quarterbacks have had seasons with fewer than 1% interceptions.  The list below is the kind of list that would show up in a record book.  

You'll see, for example, that Jason Campbell's 2008 season ranks in the top 5 all-time, and 2 other quarterbacks in 2008, Chad Pennington and Jeff Garcia, also had noteworthy seasons, with both finishing in the Top 30 all-time.  

If you look carefully, you'll notice that the list is dominated by quarterbacks in the past 2 decades.  Of the Top 50, 45 have occurred since 1990.  21 last decade and 24 this decade.  The 5 seasons in the top 50 not to have occurred in the last 19 years are Steve Bartkowski's 1983 season (#3), Bart Starr's 1966 and 1964 seasons, respectively (#7 and #18), and Ken O'Brien's 1985 and 1988 season's respectively (#36 and #38).  

So, what are we saying?  Are we saying that quarterbacks prior to 1990 were not very good?  No, not at all.  The league has changed.  From Exhibit 1, you can see that the average interception rate has been decreasing every decade.  So, we simply can't just compare a quarterback from one decade to another.  That's where Exhibit 3 comes in.  Let's take a look at that.

Exhibit 3 - Best seasons - Interceptions percentage, ranked by how different the particular season was compared to the mean, using the standard deviation as the measuring stick:

Wow, what a difference.  You can quickly see that this is a much better representation of the past 7 decades.

1940's - 3

1950's - 2

1960's - 10

1970's - 9

1980's - 7

1990's - 6

2000's - 13

Look at Bart Starr!  Ranked twice in the Top 5, and 3 times in the Top 25.  

And, this measure doesn't discount Steve DeBerg's and David Garrard's great seasons - they're both still in the Top 5.  But it does give one a bit more perspective.  In other words, this suggests that Bart Starr's 1962 season (which ranks #430 in absolute terms), when compared to everyone else's performance during that season, was better than David Garrard's performance, when compared to how all the other quarterbacks did in 2007.

Now for a little math.  Why are we using standard deviation as a measure of separation?  And why does using it make comparing quarterbacks across years more meaningful?  

A non-technical definition of the standard deviation is that it is a measure of the dispersion of a set of data around the average.  By dispersion we mean spread.  Knowing the average of the data, and knowing how spread the data is, we can try to determine how likely a given observed value is.  We can use this data to compare different data sets, and relate them to one another.   So for example, in a data set where the average is 5, and the standard deviation is 2, an observation of 8, would mean 1.5 standard deviations ((8-5)/2 = 1.5) above the the mean.  In a data set where the average was 7, and the standard deviation was 4, a value of 13 would also be 1.5 standard deviations from the mean.   You can now see why the use of a standard deviation could be useful in comparing different sets of data.  

As you can see from Exhibit 1, the league average interception rate has been changing over time.  In addition, while I have not shown it explicitly, the standard deviation around the mean has also been changing.  As a matter of fact, in the early 1940's, the standard deviations were quite high because there were fewer players involved, the number of attempted passes were fewer, and arguably the talent pool was not as great (i.e. passing was a fairly new concept).  

By relating a given observation of an interception rate in 1943 to the mean interception rate that year, and the standard deviation of interception rate that year, one can then compare that particular observation to an observation of an interception rate in 2003, by its relationship to the mean and standard deviation of interception rates in 2003.  If one assumes, in particular,  that in any given year, that all observations about the mean are distributed normally (i.e. a "bell-shaped" curve), then the comparisons become that much more meaningful.  For example we know that in a standard bell-curve, that approximately 68% of the observations will fall into a band +/- 1 standard deviation from the mean, and approximately 95% of observations will fall into a band +/- 2 standard deviations from the mean.  Also, if, after relating the observations in each year to each year's mean and standard deviation, one aggregates the data across all years (since the data has been "normalized" to the same scale - a number in relation to a mean and standard deviation; in the example earlier, the observation of 8 in the first data set has the same value on a normalized basis as the observation of 13 in the second data set - 1.5), then the aggregated data should look like a standard normal curve, with a mean of 0, and a standard deviation of 1.  

Well, I went through the trouble of doing that, and guess what - that is exactly what the looks like.  All 1,451 qualified passers over the 69 years from 1940 to 2008 were analyzed in this manner, and the result is in the graphical illustration below.  The average for the entire data set is -0.06 (close to 0), and the standard deviation is 1.00!  Also, it turns out that 69% of the observations are within +/- 1 standard deviation, and 96% are within +/- 2 standard deviations.  Quite a remarkable achievement!  I've graphed a standard normal distribution as well, so that you can see for yourself how close the actual data is to the theoretical curve.

Exhibit 4 - Same as Exhibit 3, except ranks the worst seasons:

Terry Bradshaw's 1970 rookie season ranks as the all-time worst.  And Vinny Testaverde's 1998 campaign is not that far behind.  Although Testaverde's rookie season was 1987, he didn't "get exposed" until 1988.  Most recently, Gus Frerotte's 2008 campaign was an absolute disaster.  There's some names on the list that I would not have expected to see (let's face it, these are the 50 worst performances over the past 69 years - 1,451 quarterback seasons) - names like Favre and Aikman, along with the aforementioned Bradshaw.  What was even more surprising to me was that they each showed up not once, but twice!

So now you've seen the best and the worst seasons.  How about careers?  Let's take a look at Exhibit 5.

Exhibit 5 - Lowest career interception rate - absolute figures - with a minimum of 1,000 passes attempted:

For reasons I discussed above, I don't like this list that much, as it is biased towards the more recent years.  But, since people like looking at these types of lists, I have included it.  Exhibits 6 (best) and 7 (worst) reorder the data after a player's career has been compared to the average during their career.

Exhibits 6 (best/lowest) and 7 (worst/highest) - career interception rate relative to league average during the same time:

First question is: why, after the long dissertation about standard deviations, am I not using that as the measuring stick?  Simple answer - it's difficult!  It was a relatively simply exercise to calculate standard deviations for each year.  And it's not that difficult to do it for a given set of consecutive years.  Where it becomes increasingly difficult is to do it for every combination of multiple years, over a 69-year period.  At some point before the 2009 NFL season, I'll have it done.  That will be a better reflection of who had good or bad careers, but, in the meantime, this will have to suffice.  It's the next best thing.  It is most certainly better than the absolute comparison shown in Exhibit 5.

Ok, on to a few observations.

Exhibit 6 is a who's who of quarterbacks in football history.  Taking out the 13 players who are either currently playing, or who have retired in the past five years (i.e. not Hall of Fame eligible), 14 of the remaining 37 are in the Hall of Fame.  Let's look at it another way.  There are 27 quarterbacks who are in the Hall of Fame, who played football post 1940.  And two of them, Arnie Herber, and Clarence (Ace) Parker, played much of their careers before 1940.  Of the remaining 25, 14 show up in Exhibit 6.  

Who are the 14?

Joe Montana (#3)

Roger Staubach (#9)

Steve Young (#14)

Sammy Baugh (#19)

Fran Tarkenton (#20)

Bart Starr (#22)

Sonny Jurgensen (#23)

Otto Graham (#32)

Dan Marino (#33)

Sid Luckman (#37)

John Elway (#39)

Len Dawson (#42)

Johnny Unitas (#46)

Norm Van Brocklin (#50)

And 2 more, Y.A. Tittle (#53), and Troy Aikman (#55) just missed being in the Top 50.  Not a bad list.

What about Exhibit 7?  3 Hall of Famers, Joe Namath, George Blanda, and Terry Bradshaw are in the worst 50 all-time, in terms of their career interception percentage as it relates to the league average during the time that they played.  

Super Bowl Squares

It's that time of the year, when seemingly uninterested people run around feigning interest in what is now a true American celebration - the Super Bowl.  This year's Super Bowl features the Pittsburgh Steelers and the Arizona Cardinals.  Not that anyone outside of Phoenix and Pittsburgh actually cares who is in it.  That however won't prevent Super Bowl parties from cropping up across the nation on Sunday, February 1st.  And, at most of those parties, someone will usually run around trying to get the last few stragglers to participate in the second greatest tradition of the Super Bowl - the "Super Bowl Squares" (the first has to be watching the Super Bowl commercials).  

Basically, for a small donation, you get to place your name in a 10x10 square grid.  For multiple donations, you may be able to place your name on multiple squares.  Before the actual game starts, #s from 0-9 are drawn randomly and placed across each of the 10 columns.  The same process is repeated to the left of each row.  Then the name of team 1 is drawn and placed at the top, and the second team is placed on the left.  Now you have a grid that has each of the possible last-digits of the scores of each team.  When the game ends, you look at the score, then look at the last digits of each team's score, and see whose name corresponds to that permutation and that individual wins a pre-determined amount of the accumulated donations.  This process doesn't have to be limited to the game-ending score.  Many variations exist.  For example, frequently some smaller amounts could be won based on the digit permutations at the end of each quarter.

This year, as I almost always do, I participated in one of these Super Bowl Squares.  I donated for 2 squares.  After all the squares were filled out, the #s were randomly drawn, the team's assigned, the coordinator of the game handed me my sheet.  I had drawn the following 2 permutations:

Arizona 2 - Pittsburgh 5

Arizona 8 - Pittsburgh 2

I promptly tossed the sheet in the recycling bin.

Later on, I decided to see for myself the likelihood of my winning.  

The analysis below shows the aggregated game-ending digit permutations and combinations for every NFL game played in the Super Bowl era, including playoff games.  That's 9,509 games!  That also means that there's a reasonable likelihood that the probabilities shown are close to the true probabilities.  As a matter of fact, every single permutation has been "hit" at least once.  A game ending in the 2 2 permutation has only happened once - on Sunday, December 5th, 2004 the Buffalo Bills beat the Miami Dolphins in Miami 42-32.

First, a little math.  I refer to both permutations and combinations.  There is a difference between the two.  A combination refers to a sequence or collection without regard to order.  A permutation is a combination with a specific order.  Here's an example.  Take what we commonly (and mistakenly) refer to as a "combination" lock.  We say to unlock the lock, "use combination 472".  Well, that's actually only mildly helpful.  Knowing those three #s alone we wouldn't be able to open the lock.  We need to know the specific order of that combination of #s.  In other words, is it 274, 247, 427, 472, 724, or 742.  So in this example, there is one combination.  There are six permutations.

Ok, now on to the tables and charts below.

In TABLE 1, I show all 100 permutations of game-ending scores.  So for example, one can see that the likelihood of the game ending with the winning team's score ending in a 4, and the losing team's score ending in a 3 is 2.94% (to see this, in TABLE 1, go down to the row with the digit 4, then across to the column with the digit 3).  So, this specific permutation has a 2.94% likelihood.  

If you didn't care about whether it was the winning or losing team that had the 3 or the 4 in the last digit, as long as there was a 3 and a 4, then look to TABLE 3.  TABLE 3 shows the probabilities of each combination.  As such, the 3 4 or 4 3 combination has about a 3.67% likelihood of occurring.  CHART 2 graphically illustrates what's in TABLE 3.

TABLE 2 is not meaningful in and of itself, but simply shows the probability of any given digit occurring (note that in this table, the percentages add up to greater than 100.00% since "any 7" will include for example, the "1 7" combination, that will also show up under "any 1").  CHART 1 simply illustrates what's in TABLE 2.

Let's take a look at what my chances are.  It's a little complicated so bear with me.  Remember, I have 2 specific permutations.  Ari 2/Pit 5 and Ari 8/Pit 2.  However, since I don't know ahead of time who will win the game, I need to average the 2 permutations that yield the 2 5 combination for the first scenario and the two that yield the 8 2 combination for the second.  I can do this by either going to TABLE 1, and adding the respective likelihoods of each of those permutations, or I can simply go to TABLE 3.  From TABLE 3 I can easily see that the 2 5 combination shows a likelihood of 0.36% (this is made up by the 5 2 permutation likelihood of 0.23% and adding it to the 2 5 permutation likelihood of 0.13%).  Therefore, the average expectation for the specific 2 5 permutation is 0.18%*.  For the second scenario, from TABLE 3, I can see that the likelihood of this combination is 0.28%, and hence the average expectation for the specific permutation is 0.14%.

* (Technically, I shouldn't be averaging the permutations expectations.  What I should be doing is weighing each permutation by the likelihood of Arizona (or Pittsburgh) winning or losing the game.  So for example, if the likelihood of Pittsburgh winning the game is estimated to be 70%, then a truer expectation for my specific 2 5 permutation might be 0.7*0.23% +0.3*0.13% = 0.20%.  However, if you assume that each team's likelihood of winning the game is close to 50%, then averaging is fine).

So there you have it, the combined likelihood that I would win ANYTHING is about 0.32% (0.18%+0.14%).  

Hence why I tossed the sheet.  Hope you have better permutations!  Good Luck!