There is the known past, clouded by randomness, and there is the unknown future, clouded by uncertainty. Now is merely the ephemeral intersection between the past and future.

-- Brian Burke

Spend any time reading a football blog and you will soon run across an argument about statistic X being a "bad stat". There is usually a lot of hyperbole, subjective judgement and anecdotal evidence with both sides claiming victory and neither any wiser. But what makes a stat good or bad?

This is the first in a series of fanposts I am writing to analyze some common NFL statistics, focusing on how much value they have regarding team wins. I want to acknowledge the work of Brian Burke, Chase Stuart and even our own Matt Grecco, who inspired this analysis and whose methodologies I have leveraged, as well as Pro Football Reference, Armchair Analysis and NFL.com as the sources of my data.

What makes a good stat?

As you can tell by this post's title, passing yards (and by comparison, all volume yards) is up first. I selected this, not because it is necessarily a contentious metric, but rather because it is ever-present. It is ingrained into how we measure QBs, rank offenses, determine MVPs and decide Hall of Fame entrants.

Perhaps more importantly though, is that it is an easy one to use as an introduction to my analytical methodology.

Using data(1) from 2000 - 2016, the Colts rank 2nd in passing yards per game (268.8) and 8th in passing yards against (227.8). The differential of 41 yds per game ranks 1st in the NFL (although I don't believe we hung a banner for that).

Limiting the data to just the Luck era, the Colts have the 7th highest passing yards per game (272.6) but also the 23rd worst passing yards against (258.1). That differential ranks 11th.

Unlike other metrics like drops or inaccurate throws, passing yards is a straight forward objective stat. It isn't opinion. It isn't complex and it doesn't change over time. It is also publicly available and can easily be obtained from multiple sources. These are all excellent qualities.

However, just because a stat has a lot of good qualities doesn't mean it has value. Metrics should do one of two things:

1. Explain the past
2. Predict the future

Great stats do both, while bad stats do neither. Below are graphs plotting cumulative Win % against Passing Yards for the offense and defense (the blue dot is the Colts).

For the offense, there definitely is a trend that more passing yards equates to more wins, but it doesn't appear to be very robust. On the defensive side, the trend line is flat suggesting nothing about wins at all.

This is our explanation of the past and it can be measured with correlations.

Explanatory

Breaking the data down into individual season totals, yields the following overall correlations:

(Quick lesson in correlations: The magnitude indicates how strong a relationship there is between two variables. The sign indicates if the numbers move together or in opposite directions.)

Right off the bat, passing yards has a near zero correlation to wins and so tells us nothing about Win %.

But what about the offense? Is 0.28 a "good" correlation to wins? Adding in rushing and total yards (2) gives us a sense of how relatively strong the passing yard correlation is.

Passing yards look fairly weak, while total yards has a much stronger correlation, which makes sense because it is a more comprehensive number.

And since rushing yards appears to be a better explanatory metric, that means teams should focus primarily on running, right?

Not so fast Kowalski. We use correlations because they describe the relationship between two variables, but what they don't tell us is if one causes the other.

Let's look at the same data by quarter (non-cumulative) to see if these correlations hold throughout the game.

For 3 quarters, passing is much more associated with wins than rushing. But then, magically, in the 4th quarter, passing yards throws it into reverse and is linked to losses.

This is what is meant when you hear the mantra "correlation is not necessarily causation". More passing yards does not literally cause losses, but losses (or trailing late in the game) can certainly "cause" lots of passing yards. And while first half passing may explain wins, it gets lost in the game totals.

Similarly rushing more doesn't cause wins but rather grinding out the clock with a lead causes rushing yards. So, even though rushing has a higher overall correlation to wins than passing, it doesn't truly explain wins very well. So be wary of anyone who says running is the "key" to wins. It isn't.

Predictive

So, we know that passing yards isn't very informative from an explanatory basis for either the offense or defense. But how well does it predict wins?

If wins are truly driven by a statistic, then wins should generally be driven by that statistic in all games, not just the games being measured.

Taking the data, I randomly (3) divided each team's season into two 8-game semi-seasons. I then calculated metrics for each team's semi-season and compared them to the win percentage from that team's other 8 games. Confused? Maybe a picture will help.

I am still using all the data, but instead of measuring what happened in a game (explanatory), I am trying to guess what will happen in other games (predictive). The idea is that if a metric causes wins and is repeatable (a skill) then it should cause wins in the other games too.

Now I will caveat this by saying that predicting the future is inherently difficult and so the predictive correlations are expected to be lower, but if they are dramatically lower then that means not only is the predictability poor but the explanatory power was probably overstated as well.

To start, I re-calculated the explanatory correlations using an 8-game summary(4) instead of 16. I then determined the predictive correlations and compared the two. Here are the results for the offense:

Passing yards stayed flat, meaning it's lack of information is just as unhelpful for predictive purposes.

Rushing yards fell off a cliff, just like when we looked at the data by quarter, which confirms that rushing doesn't "cause" wins at all.

Also, the 0.33 predictive correlation for total yards may seem low but that is actually a pretty good number. This runs counter to a lot of of what I read on blogs that says total yards are worthless.

For the defense, correlations are typically negative meaning the more yards a team gives up the fewer wins it will see. But I have reversed the sign here to make the graphs easier to read:

Passing yards remains worthless and note that like the offense, there is a similar drop-off in rush yards against, meaning a lot of its correlation was likely due to 4th qtr strategy changes.

Team

Even though the offense and defensive passing metrics are bad, perhaps combining them into a team differential will improve things.

Nope. Same story with regards to passing yards: important to win the passing game early, but the 4th quarter masks that value and so passing yard differential has poor explanatory power which carries over to poor predictive power.

. . . oh, and rushing is over-rated.

Alternative Metrics

So, passing yards, passing yards against and passing yard differential are all bad stats: they are neither explanatory nor predictive. But we saw that passing is important early in the game so how can we measure passing as a driver of wins?

The answer is efficiency stats. Below is the data for offensive passing yards and offensive yards per attempt (YPA).

The YPA correlations to wins are similar to that of passing yards for the first 3 quarters but does not get diluted by the 4th quarter noise. This makes it a far better explanatory stat.

And while game level YPA loses a lot of its power for predictive purposes, it still ends up as a strong predictive stat; much better than total passing yards.

That is why YPA and many related stats (NYP/A, ANYP/A, passer rating etc.) are so prevalent. Passing efficiency explains and predicts wins A LOT more than passing volume.

The same story generally holds true on the defensive side and with team differentials:

SUMMARY

I know this was lot to take in so let me give you the TLDR version:

• Total passing yards is a bad stat. There is too much 4th quarter noise to give it any value when measuring team, offense, defense or QB performance.
• As an alternative, YPA, YPA against and YPA differential are all very strong metrics to measure QB, offense, defense and team performance. If you use those you really can't go wrong.
• The conventional wisdom is that Total Offensive Yards is a bad stat but it actually is quite powerful at explaining and predicting wins even more so than YPA.

I will wrap this up with a chart of all the metrics presented, ranked in order of their predictive power and suggestions as to which metric to use when measuring different phases of the game:

• Team Measure: YPP differential is the best metric presented here for judging team performance but Total Yard differential and YPA differential are pretty good too.
• Offense Measure: It's a toss-up between Total Yards or YPP as they both provide good value but Total Yards is the more commonly available stat. YPA is a close third.
• QB Measure: YPA is the best most direct metric presented here to judge QB performance
• Defense Measure: Passing yards against is the worst stat you can use to grade a defense. Instead, use just rushing yards but as that is not really an intuitive stat for many people use YPA against as a good common-sense substitute.

FOOTNOTES:

(1) Unless otherwise noted all numbers presented are from 2000 - 2016 regular season data

(2) Total yards include sack yards

(3) Each game in a season was given a random number and assigned to one of two 8-game semi-seasons. The analysis was run 3 times and the results averaged to get the final correlations presented.

(4) This will cause the numbers to be a slightly lower due to increased variance from the fewer games (noise is relatively louder).