Tenacious D: Does Defense Really Win Championships?

The stage for Super Bowl 52 has been set, which means it is time for the football ideologues to start chanting their favorite mantra of “defense wins championships.” It’s a trope turned gospel through sheer repetition. But as tedious as these zealots are, their words are not without merit.

They will point out the fact that in the 9 Super Bowls, where the #1 scoring offense has met the #1 scoring defense, it is the defense that has walked away with the Lombardi 8 times. They’ll rightfully tell you that championship defenses, like the Bears 46 or Tampa 2, evolved the game. They’ll remind you that the elite defensive squads have such a place in championship lore as to warrant their own names; the No-Fly Zone, the Steel Curtain, the Legion of Boom, the No-Name defense.

Are these merely dogmatic platitudes or is it truly NFL wisdom? Well, Betteridge’s law aside, the folks at freakonomics.com published an analysis 6 years ago to resolve this very issue. So, if you want to know the answer go there. NO, WAIT!!

Stay here, ‘cause I’m going to do mathy things with graphs.

In that article, the authors didn’t publish their methodology nor provide detail of their results, which leaves me (and I’m sure, you) with unanswered questions. As such, I’m going to create my own little study right here.

This is not the greatest analysis in the world. This is just a tribute.

METRICS

The first step is to identify the appropriate metric to measure offensive and defensive performances. If I calculate differentials in expected points per play, map that against score discrepancy and regress that against time . . . Zzzzz.

OK, let’s just use points and points against.

Points is a highly predictive metric of wins but it isn’t without its problems. I’ve added some tweaks, like adjusting for opponent, to address these deficiencies and have outlined my process in the footnotes(1).

Using game level data, I calculated season averages and rankings for each team’s offense and defense for all 51 years of the SB era. The table below lists the number of SB wins for top 10 offenses or defenses by position rank.

#1 defenses have won 9 Super Bowls and #1 offenses only 6, but the gap doesn’t get much bigger than that. Cumulatively, the Top 10 defenses have won 40 times or 78% of Super Bowls, while top 10 offenses have accumulated 38 wins for 75% of the total.

Wait, isn’t that 153%?

Yep, sometimes the winner has both a top 10 defense and offense. That has occurred 30 times, which is about 60% of all winners. If you remove those years, that leaves only 10 instances where a top 10 defense won without also having a top 10 offense.

The point is that having a top defense is a good thing and having a top offense is a good thing, but most winners have both.

RATINGS

In the chart below, I have plotted the team production for all 102 Super Bowl participants. Production is measured in adjusted points per game relative to league average. Offensive production is points over average (OPOA) while defensive production is points allowed under average (DPUA).

OPOA is plotted on the vertical axis and DPUA on the horizontal. Each red dot represents a team that lost the Super Bowl while each green dot is a Super Bowl winner. The blue lines are the median values, meaning that half of the teams lie above/below the horizontal blue line and half of the teams lie to the left/right of the vertical blue line.

If defense truly wins championships, then a majority of the green dots should be to the right side of the chart and a majority of the red dots should lie to the left. And that appears to be the case.

Here are the counts by each quadrant of the graph.

You can see that the 51 teams in the top half of defenses have a 31-20 Super Bowl record (61% win rate). Conversely, the record for teams with top half offenses is only 25-26 (49% win rate).

So does that mean that the D rules? Weeeeell, kinda sorta, but not really.

SB winners, in general, do have statistically better defenses (2) than SB losers and conversely, their offenses are not significantly different.

However, a 6 game difference between defensive winners and offensive winners just isn’t that strong of a result. A 61% win rate is not significantly different from 49% with such a small sample, although it is very close(3).

What you can be sure of though is that teams with a top half defense and offense are 13-7 (65%), while teams in the lower half of defenses and offenses are 8-12 (40%). That is a significant result.

MATCH-UPS

One of the weaknesses of the analysis so far is that it doesn’t measure head to head match-ups. What happens when a good defense meets a good offense? In the upcoming Super Bowl, NE and PHI both have a top 10 DPUA. So, no matter who wins, the case is made for a good defense winning. A better comparison is to see if the eventual winner’s defense was rated higher than the losing offense.

I can calculate the difference between the SB winner’s OPOA and the loser’s DPUA to determine the offensive differential (>0 = off better than def, <0 = off worse than def). The same method can be done for the winner’s defense (winner DPUA - loser OPOA)

These differentials are plotted below:

Teams with defenses better than the opponent offense are on the right half of the graph and only 24 of 51 winners are found there. Comparatively, the upper half of the graph is where the offense was better than the opponent defense and that logs only 5 more winners with 29 total.

If instead of offense vs defense, I were to plot the def vs def and off vs off differentials, the story is similar. The better defense of the two teams wins 32 times and better offense wins 28. That is a Matty Ice sack and Santonio Holmes toe tap away from a push.

The statistical term for all of this is “coinflippish”; as in having all of the qualities of coinflippishness.

CONCLUSION

These results shouldn’t be surprising. It is true that good defenses in the NFL win a lot of championships, but that completely ignores the fact that good offenses do too. The vast majority of these teams all have good offenses and defenses else they wouldn’t have made it to the championship.

The marginal differences in play are often small and although narratives are popular, no one component, like having a good defense, is the determining factor for victory. At the end of the day, there just isn’t any evidence that clearly favors a good defense over a good offense.

Which brings us to the upcoming Super Bowl. Both teams ended the season in the top 10 for OPOA and DPUA, but NE has got the edge for both.

In the 19 Super Bowls, where a team had both the better offense and defense, that team won 14 times for a 74% win rate.

Crap.

FOOTNOTES

1) Adjusted point methodology

• Points are calculated using regular season games only. No special teams scores are included. All TD’s assume a successful 1 pt conversion.
• (OP): Offensive points = FG*3 + Rush TD * 7 + Pass TD * 7.
• (NOP): Net OP = OP - Int TD * 7 - Fumble recovery TD * 7 - Safety *2.
• (NOPOA): NOP Over Average = NOP - League Avg NOP for season.
• NOPOA is adjusted against opponents within a season using an iterative feedback convergence methodology to create Adj NOPOA for offenses and Adj NOPOA against for defenses (Thanks for the code Matt Grecco!) .

2) t-test for Paired Two Sample for Means with an hypothesized difference of 0 alpha level = 0.05, p-value = 0.01

3) Two tail binomial test alpha level = 0.05, p-value = 0.052