Passer Rating and Other Stats I Hate

This is the 5th in a series of articles I am writing to analyze some common NFL statistics, focusing on how much value they have relative to team wins. I want to acknowledge the work of Brian Burke, Chase Stuart and even our own Matt Grecco, who inspired this analysis and whose methodologies I have leveraged, as well as Pro Football Reference, Armchair Analysis and NFL.com as the sources of my data.

---

There are a lot of things not to like about passer rating.

• It's old. It has not been updated since the Vietnam war (literally!).
• It's range has no intuitive meaning, scaled from 0 to a cryptic 158.3.
• And then of course there’s this: Passer Rating = 100 *(((MAX(0,MIN((Cmp/Att - 0.3) * 5, 2.375))) + (MAX(0,MIN((Yds/Att - 3) * 0.25, 2.375))) + (MAX(0,MIN((TD/Att) * 20, 2.375))) + (MAX(0,MIN(2.375 - INT/Att * 25, 2.375)))) / 6)

Ooph.

At it's heart it is good idea: take 4 metrics, by which a QB is measured (Completions, Yards, TDs and INTs), turn them into efficiency stats and average them together. That's a great thought.

But as with all great thoughts, it is the execution of them that matters most and that is where passer rating goes off the rails. Here’s an interesting read about the origin of passer rating and in it you can see some of the missteps taken along the way.

But despite it's many many flaws, passer rating is not an entirely bad stat. I'm not saying it's the standard by which to measure a QB, but it certainly is a serviceable stat. It's a fixer-upper. So let’s fix it.

THE RANKINGS

Since Luck joined the Colts, the team passer rating has been a disappointing 86.4 which is only 18th place.

That is only slightly better than the defense who comes in at 19th place in passer rating against.

Basically, passer rating says our offense and defense perform at similar levels. Hmmmmmmmmm.

THE NUMBERS

Passer rating is strongly correlated with past wins and has very good ability to predict future wins as well (>0.30 is good). This alone puts it far above other stats commonly used to measure QB's like passing yards or TDs. The reason it performs so well is for two reasons.

One is, it's an amalgam of some good stats. This is not always a good thing as sometimes less is more, but in this case, more is more.

Another reason for its effectiveness is that it's an efficiency stat, utilizing per attempt measurements instead of volume metrics, which are more susceptible to the noise of 4th quarter strategy changes.

COMPONENTS

Breaking passer rating into its four individual components of completion %, yards per attempt, TD rate and INT rate allows a better view of what drives its correlation to wins.

If you have read my previous articles, then you have already seen 3 of these. Y/A and TD/A stand out as having both good explanatory and predictive power. INT/A though, is a mixed bag.

When I wrote about TD/INT ratio, I showed that INT rate was good at explaining wins (throw picks and lose) but bad at predicting them because they are random and game situational (the game was already lost when the pick was thrown). This means if you want an explanatory stat then INT rate is good, but you wouldn't want to include it for predictive purposes.

And that leaves CMP%, which neither explains nor predicts wins very well. Additionally, it is a strange thing to include as it is already accounted for with Y/A:

Y/A = (Yds / Cmp) * (Cmp / Att)

In effect, CMP% is a detailed factor of Y/A and as often happens in statistics, the more granular the view, the more noise you are likely to see.

Some may be confused by how combining two stats, each which are bad at predicting wins, can result in one stat that is good at predicting wins. Think of it this way:

• Yards matter.
• CMP% suggests yards (given a high Yds/Cmp)
• Yds/Cmp suggests yards (given a high CMP%)
• Y/A actually IS yards (unless you have a very low number of attempts which is a whole ‘nother article).

Since CMP% is already a component of Y/A, it is redundant and so adds no value to passer rating. When the music stops, this stat will have no chair.

SACKS

One common complaint about passer rating is that it ignores the impacts of sacks.

Since 2012, Colin Kaepernick ranks 18th in passer rating(1). But he was also one of the most sacked QBs in the league, with a sack rate of 9.2%. If lost yardage and the attempts from a sack are added into the traditional passer rating formula, Kaep drops 10 spots to 28th.

OK, the impact of sacks can be large, but aren’t sacks the fault of a weak O-line? Yes and no.

Analysis shows that when a QB switches teams one of the most persistent stats that follows him is sack rate and when different QBs play behind the same O-line, sack rates are dramatically different. The data strongly support that in large part, sacks are the result of QB play.

In and of itself, sack rate is not very correlated to wins (-0.26 explanatory, -0.21 predictive). But as was previously demonstrated with CMP%, when combining stats, the whole can often be greater than the sum of the parts.

Incorporating sacks into Y/A yields the advanced stat Net Yards per Attempt:

NY/A = (Yds - Sck Yds) / (Att + Sacks).

And incorporating sacks into TD rate yields . . . actually, I have no idea what it yields because I have never seen it calculated it before.

This illustrates what most stat geeks already know, which is NY/A is a far superior stat to Y/A. And since TD rate is improved when adding sacks, we can safely say that sacks matter (2).

COMPONENT VALUATION

An additional criticism of passer rating is that it does not equitably value each component in relation to the actual game impact. If you ignore the minimum and maximum thresholds, the passer rating formula can be re-arranged to look like this:

Passer Rating = 2.08333 + 83.33 * CMP% + 4.1667 * Y/A + 333.33 * TD/A - 416.67 * INT/A

Looking at the formula in this way, you can see inherent equivalences between the components.

For example, the multiplier for INT rate of 416.67 is 100 times larger than Y/A (4.1667). This means that given the same number of attempts, passer rating is indifferent between 100 fewer passing yards or 1 more interception. In other words, an INT is equal to 100 yards.

Whaaaat?! That is more inflated than a Patriot football before a bathroom break. Similarly, passer rating gives TDs a value of 80 yards (333.33 / 4.1667).

The impact of these overvaluations can easily be seen by breaking apart the average passer rating score into it's separate component values.

Since 2000, half of a QBs passer rating has been comprised of TD/A and INT/A. Also, another 30% is CMP%, which has already been determined to add no value whatsoever. That leaves Y/A, the best stat of the 4, with only 20% of the value.

An additional problem here is that the formula relies on inputs developed in 1973 when football was a far, far different game than today. Even within just the last 10 years, the average passer rating has increased by about 10 points.

UNITS

Another issue many people have with passer rating is that the numbers have no intuitive meaning. Last year Andrew Luck posted a 96.4 passer rating. But what does that actually mean? 96.4 units of what? QB goodness?

Trying to come up with a meaningful scale is problematic when combining disparate stats like TDs and yards. How do you add 2 TDs to 250 yards? Does that equal 252 units of something?

In the 1988 book "The Hidden Game of Football", the authors popularized the idea of yardage equivalents for TDs and INTs. The numerical support for this can get complex, but think of it like this: how many more yards would the average drive require to be a touchdown? Or how many yards of potential field position does a pick cost?

Previously, I showed that inherent in passer rating, there is an 80 yard equivalent for a TD. The Hidden Game authors came up with their own value of 10 yards, but Chase Stuart has updated that value to 20 yards, which is widely accepted today. Similarly, a value of -45 has been accepted as the equivalent yardage impact of an interception.

Using yardage equivalent values like these, stats can be converted to a common unit (yards) and easily combined resulting in more intuitive measures.

ALTERNATIVE STATS

So, putting this all together, passer rating would be improved if it was a simple formula, based on updated numbers, that ignored CMP%, incorporated sacks, didn't overvalue TDs or INTs, and combined them all into a single stat with an intuitive unit of measure that isn’t re-scaled to an enigmatic range.

So is there some magic stat that does all that? Yep.

Adjusted Net Yards per Attempt is an advanced stat defined as:

ANY/A = (Yds - Sck Yds + 20 * TD - 45 * INT) / (Att + Sck)

This translates TDs and INTs into yardage equivalents and then adds them together with sack yards making a single efficiency stat. Here is a how the average individual component valuations of ANY/A compare to each other.

Much better. Y/A is prominent as it should be and TDs/INTs have more relavant values. The scale even makes sense as it is all in terms of yards per attempt . . . but with sacks . . . and adjusted . . . you get the idea.

To focus more on predictability, simply remove the INT portion from the formula. I've never seen that resulting stat discussed before so it doesn't have a name that I know of, but I'll call it semi-ANY/A:

semi-ANY/A = (Yds - Sck Yds + 20 * TD ) / (Att + Sck)

ANY/A is a more predictive metric than passer rating with similar explanatory power. For semi-ANY/A there is less explanatory power because INTs, which explain win/losses, were removed, but there is more predictive power because INTs also tend to be unpredictable.

Therefore, ANY/A should be used to assess a QB’s past performance and semi-ANY/A to guess their future. Both of these measures are waaaaaay easier to calculate than passer rating and all the data you need can be found in each game’s box score.

RE-RANKING

When re-ranking teams by ANY/A, the Colts offense moves up 5 spots to 13th, which is more in line with other advanced metrics I have seen.

The defense drops 5 spots to 24th. Yeah, that feels like home.

SUMMARY

• Passer rating is old and busted jawn
• ANY/A is the new hotness

---

FOOTNOTES

1) Regular season only. Minimum of 200 attempts.

2) Since correlation is not causation, we can’t say that sacks specifically cause wins. It is more likely that a higher level stat like “QB pressures” would show more win correlation. But I don’t have that data.

Unless otherwise noted all data was from regular season games from 2000 - 2016.

Explanatory correlations are calculated by metric totals by team per season against win% by team per season.

For predictive correlations, each team's 16 game season is randomly divided into two 8-game semi-seasons. The correlation is calculated between metric totals from one semi-season to the win% from the other semi-season. The idea is that if a metric causes wins AND is repeatable then a team that is "good" at the metric in 8 games should replicate that success in the other 8 games.

Previous Analysis - Passing Yards

Previous Analysis - TD/INT Ratio

Previous Analysis - Time of Possession

Previous Analysis - Third Down Efficiency