QB Stats Matter (pt.2)

This article uses statistics pulled from Football Outsiders and the nflscrapR project.

---

If you haven’t read my last article about why QB stats matter, then let me get you up to speed.

1. QB efficiency stats are critically important in assessing QB talent.
2. See point 1

A QB’s efficiency directly translates to the ability of an offense to move the ball and score. That really is a no-brainer. A QB that averages more yards/value per play, will have an offense that converts more first downs and scores more points. This is easily demonstrated by the following graph from my previous article, measuring EPA per dropback against points per drive. (actually this isn’t the exact graph I showed as votingmachine shamed me into upping my data visualization game)

This shows the strong relationship between QB efficiency and points scored. The R-squared value of 0.9198 means that almost 92% of points is explained by efficiency(1). So for people who are prone to claim that stats don’t tell the whole story, they are right: in this case, they only tell 92% of the story.

So why are so many people dismissive of QB efficiency?

---

WINS

The main argument that I hear from stat-Luddites is that a good QB elevates their team, earning wins beyond any measure and conversely, a bad QB can drag their team down, while still maintaining good stats.

Football is a sport intrinsically connected to itself, so I don’t disagree at all with the premise that a good QB makes everyone better: a rising tide will lift all boats. However, I absolutely do not believe with the implication that a rising tide cannot be measured.

I do get why many people disagree with me, though. The following graph shows win rate against QB efficiency and seems to support their case.

This graphically represents some of the typical “winner” and “loser” QB narratives. Tom Brady has great stats but he also just gets far more wins than those stats dictate because reasons. Philip Rivers, on the other hand, can barely win half his games while he pads his stats to make himself look good.

I have written previously why this kind of comparison is ridiculous, primarily because it completely ignores the impact of a team’s defense. Wins just aren’t a QB stat. Of course, that is not going to appease my detractors, so I’ll have to include wins in my analysis.

Since a team’s offensive points are primarily a function of QB efficiency and since a win is simply defined as point differentials, then it follows that wins should strongly relate to QB efficiency differentials. As such, the following chart compares win rate against EPA/db differential (team EPA/db less opponent EPA/db) .

The increased R-squared (0.8486 vs. 0.6138) shows that wins are explained far more by QB efficiency differentials than QB efficiency alone. This is an important result and it directly parallels experience. Of the 2,816 regular season games played since 2009, almost 82% of them have been won by the team with the higher EPA/db. Notice also that the top 10 teams by passing differential in this chart represent 9 of the last 11 Superbowl winners and 15 of the 22 participants.

Yet, even with this measure the Chargers are still a glaring outlier. So does this indeed show that Rivers is under-performing when it comes to wins? I would argue the opposite. Rivers isn’t as much below the line as he is to the right of it. This doesn’t show him pulling down a team that would otherwise win 62% of their games, rather it shows he is on a bad team that he is dragging to a 51% win rate.

How do I know that? Because I can data.

---

SUPPORTING CAST

The previous chart shows a 0.115 EPA/dropback differential for the Chargers which is good enough to rank 5th of all teams. Deconstructing that, we can see that Rivers’ arm is the primary driver of that ranking.

In the following set of charts, the black bar represents the efficiency differential while the red and green bars are the offensive and defensive components. In other words: green - red = black.

The Chargers 0.159 EPA/db passing efficiency is the 3rd best since 2009 (shown as the bold green dot on the right chart) and Philip Rivers started every single one of those games. Over that time, the cumulative defensive passing efficiency was pretty much just average ranking 14th (red dot near the middle).

The story is very different when looking at the rushing game though.

It’s common for teams to have negative EPA per carry — all but one team in this study is < 0. However, the Chargers offense run game efficiency is really low, ranking 29th among all teams. Again, the defense performs just slightly better than average (14th) so its impact on the differential is negligible, ranking 29th.

That’s really not helping the passing game and the story gets worse with special teams play.

The Chargers’ special teams rank 28th in EPA per play, while their opponents ranked 4th. That’s not a good combination and results in the 2nd lowest differential of any team(2).

Football Outsiders data also agrees with this analysis. Since 2009 the average DVOA for the Chargers has ranked:

• Passing - 3rd
• Rushing - 28th
• Special Teams - 32nd
• Defensive - 20th (18th passing, 29th rushing)

So what happens, when you combine great passing with terrible rushing, horrible special teams and a bad defense? You get a 51% win rate team with a great quarterback.

Let’s look at win rate comparisons again, but this time include impact from rushing and special teams.

More of the win rate is explained (0.9176 r-squared) when looking at total play efficiency. As such, the Chargers numbers are much more in line with expectations, which means the low win rate was primarily a function of poor performance independent of Rivers.

Notice that the Patriots are now also much closer to the expected win trendline, exceeding expected wins by only 1.2%, which equates to 2 games over 11 years. Their defense was right about average according to DVOA (16th) but both their run game and special teams ranked 2nd, boosting their win rate above Brady’s impact alone (well Brady and Matt Cassel).

---

CONCLUSION

When it comes to scoring points, QB efficiency, specifically EPA/db, is incredibly important. Not only does it explain offensive results well but it also predicts future production better than almost any stat. QBs are, and rightly should be, judged by this metric.

However, when it comes to actually winning games, it’s a team effort. Most teams have a lower overall team efficiency (EPA/play) than they do passing efficiency (EPA/db), making a sort of efficiency gap: that’s just the nature of the game. However, some teams gaps are much larger than others demonstrating a significant difference between the level of QB play and the rest of the team, often resulting in good QBs losing a lot of games.

The top 3 teams in efficiency gap since 2009 are the Chargers, the Falcons and the Saints. All are teams that have had their share of sub 0.500 seasons and it’s not surprising to me that I have heard Philip Rivers, Matt Ryan and Drew Brees all labeled as QBs who are less than their numbers. I think the more supported argument is that they are really good QBs that have sometimes been on really bad teams.

---

FOOTNOTES

1. 0.9198 R-squared for EPA per dropback at the team level for cumulative data between 2009-2019. The QB specific R-squared in that same time frame is 0.8710 for QBs with >=500 attempts. The QB specific EPA/db by season R-squared for QBs with >=200 attempts is 0.8137
2. “Team” for special teams is defined as the team punting, the team receiving kickoffs, the team executing field goals and extra points, and the team attempting 2 -point conversions. “Opponents” are the reverse side of those situations.