QB Stats Matter (pt. 3)

This article uses statistics pulled from Football Outsiders, the nflscrapR project and Pro Football Reference

---

Once more unto the breach, dear friends, once more. This is the third in a series of articles that I have written addressing QB efficiency. In case you haven’t read the first two and want to get caught up you can find them here: pt 1, pt 2.

In the comment section of the last article, Clydesdales asked,

Maybe some stats on one guy in particular could help me wrap my head around stat’s correlation to wins. In my opinion, John Elway is the best QB I have ever seen lace them up. . . . What would his stats tell me about him?

Great question. Clydesdales is not alone in his opinion of Elway. He is broadly considered one of the best QBs to ever play, easily in the top 5 list for many fans. Evidence to back these opinions are often in the form of wins:

• 2 Superbowl wins
• 5 Superbowl appearances
• 5th most career wins by a QB
• 0.645 career winning %
• 0.667 playoff winning %
• 40 game winning drives (7th most)
• 31 comeback wins (6th most)

Others point to his accolades or career production

• NFL Player of the year
• NFL MVP
• SB MVP
• 9 Pro-bowls
• 51,475 passing yards (9th most)
• 4,123 completions (10th most)
• 300 passing TDs (12th most)

That’s certainly an impressive resume. However, none of that really describes anything about his actual play. It certainly describes someone who played a long time (lots of volume), had great team success (lots of wins), and was well regarded (voted for lots of awards), but it doesn’t actually measure anything specific about his value as a QB.

Assessing the contributions of a QB legend is the perfect opportunity for me to demonstrate the points I have been trying to make.

---

METRICS

I have spent the previous 2 articles attempting to illustrate 3 main points:

• QB passing efficiency is strongly related to points scored.
• QB performance (and offensive points) are far less related to wins.
• Overall team efficiency is strongly related to wins.

The corollary here is that win rate is not a QB stat. I know there are people that reject this out of hand, but it really isn’t up for debate: the data could not be more clear. Certainly, QBs have a significant impact on wins and all things being equal, a good QB will get more wins than a bad QB, but all things are never equal. Defense, rushing and special teams add large variances on top of a QB’s play, dramatically impacting game results. That is why great QBs like Drew Brees have had multiple sub 0.500 seasons and why Tom Brady never has (yet).

Measuring Elway, however, introduces the variable of time. Does 1980s-90s football have the same relationship of QB efficiency to points and wins? The numbers from my previous analysis measured QB efficiency using expected points added (EPA), but unfortunately, my EPA data only goes back to 2000. So, if I am to answer that question, I’ll need an alternative measure.

That measure will be to modify the existing stat of adjusted net yards per attempt (ANY/a). For those unfamiliar with ANY/a, I wrote about it in detail here, but it’s just a fancy yards per attempt that includes passing yards, sack yards and bonus/penalty yards for TDs and INTs.

I’m modifying it because, I want to add QB scrambles, but since my pre-2000 data does not give me that detail, I simply included all QB rushing attempts. This will necessarily include designed QB run plays — which I’d rather not do — but beggars can’t be choosers. I’ll call the resulting stat, total adjusted net yards per attempt (TANY/a):

TANY/a = (passing yards - sack yards + QB rushing yards + 20 x passing TDs + 20 x QB rushing TDs - 45 x interceptions) / (attempts + sacks + QB carries) (1)

I showed previously where a regression of EPA/db against points per drive (PPD) had an r-squared value of 0.9198(2), which means almost 92% of PPD is explained by QB efficiency. To see if that holds up over time, I did the same math by decade comparing TANY/a to PPD(3) and win rate. Here are the r-squared values for each.

The results are fairly stable over time and they completely agree with my previous analysis:

• points are a QB stat: efficiency explains about 90% of points.
• wins are not a QB stat: efficiency/points explain only about half of wins.
• wins are a team stat: efficiency differential explains about 80% of wins.

So while TANY/a is not as good as EPA, it is still a very strong tool to use even for games played 35 years ago.

---

RANK

Elway’s career spanned 1983 - 1998. So, I took all regular season games from those years and calculated cumulative TANY/a for the top 100 QBs by attempt volume. In that list, Elway finished 15th with a TANY/a of 5.6.

Now, some of you are already shouting at your screen, so let me clarify that this is not meant as a strict ranking of QB talent. There are so many variables involved in QB play that a single stat cannot possibly capture it all (let alone a yards based one).

Also, this encompasses Elway’s entire career, which is being compared against what may be small portions of other QBs’ careers. So, while there are some names ahead of him that may make sense (Montana, Marino), there are others, like Doug Flutie, whose numbers are basically a season and half outlier.

The only thing a 15th place rank tells me is that out of 100 QBs, Elway was near the top. To gain more insight, I broke the data out by year.

The blue line is his efficiency numbers (measured on the left axis) and the gray line is the points per drive he put up while under center (right axis). The dashed black line is the NFL average for that period (both axes). In 1983 - 1992, he often had good efficiency but with a lot of variability. On the other hand, from 1993 - 1998, he was consistently great.

Like most QBs, he improved after his first few years in the league. 1986 was a good efficiency season and the Broncos made the Superbowl. By 1987 he looked to be the next great QB, with an outstanding season that earned him the league MVP and a 2nd Superbowl trip.

However, the next 5 years saw a step back in efficiency and point production, with the team missing the playoffs 3 of those years and being on the losing end of history’s largest Superbowl blowout with one the worst QB performances ever (10 points, 78 net passing yards, 2 picks, 0.5 TANY/a). In 1992, he was just plain bad, finishing 24th in TANY/a and 22nd in PPD out of 28 teams.

In that 10 year period, Elway could only manage a 54.7% completion rate and 3.6% TD rate, both ranking 22nd out of 28 teams but his overall efficiency was lifted by a 9th best INT rate (3.6%). His mobility also helped his numbers with an 11th best sack rate (7.6%), 5th best QB yards per carry (4.6) and an additional 22 rushing TDs (8th most). His efficiency was more about his legs than his arm.

Prior to the 1993 season, his legacy was still in doubt and the Broncos were planning for the future having just drafted a first round QB. Then the 1993 season happened and Elway found his next level. For the next 6 years he played at an extremely high level.

Notice that over his career, Elway’s efficiency was strongly linked to his points per drive (correlation = 0.89). The same can not be said for win rate (correlation to points 0.34)(4) .

---

A TALE OF 2 ELWAYS

Splitting his career into the periods of 1983-1992 and 1993 on, his stats look like this:

Based solely on 1993 forward, stats guys and non-stats guys alike would call that an elite QB: huge playoff success, huge regular season win rate, large point production and high efficiency in the top 3 of all QBs.

However, between 1983 -1992, there is a disconnect in the story. Out of the top 100 volume QBs, Elway’s regular season win rate was a 10th best 61.8%, but his efficiency (and points) only ranked 39th.

I’ll illustrate the point . . . literally. This chart shows that Elway’s efficiency perfectly explains the Broncos point production (i.e. Denver is sitting right on the trendline).

Yet this next graph shows their win rate was far above what offensive point production alone would suggest. Based on points alone, they should have won only about 50% of their games not 62%. So if the wins didn’t come from points, then where did those extra wins come from?

It’s no real mystery. When Elway was under center during that time, Denver’s defense gave up the 6th fewest points per drive to opponents. Therefore, when expanding the measure of efficiency to include the defense, along with rushing and special teams (5), the win rate falls in line with the data and the disconnect is resolved.

The “extra” wins came from the performance of the rest of the team.

---

DRIVERS

Before 1993, Elway was above average in efficiency, but no where near as good as the last 40% of his career, when he was one of the best. Showing the disparity of those years on a standard normal curve may make it easier to understand the scope of the improvement — unless you don’t know what a standard normal curve is, I guess.

De-constucting TANY/a into it’s major components shows that he improved in just about everything: he was much more accurate, he took fewer sacks, he threw far more TDs and fewer picks. He was simply much better across the board, becoming an elite passer.

---

CONCLUSION

The original question was, what does Elway’s stats tell about him? To me, they say that relative to his peers, in his first 10 years he wasn’t a great passer but his mobility made him an overall good QB and he played on a team with a defense that allowed for a lot of wins.

In the latter part of his career, the stats make it clear that he was fantastic passer, getting better as he got older and leaving at the top of his game. In addition, over that time, his team’s rushing game ranked #1 in DVOA, special teams ranked 8th and the defense was still good at limiting opponent points (10th). Thus, with an extremely strong team he enjoyed even better post-season success.

Additionally, Elway’s early, limited efficiency might not have been his fault. Chase Stuart wrote an article incorporating numbers to argue that his receiving corps was to blame. Here’s an excerpt:

For over a decade, Elway was dealing with a pretty mediocre set of receivers, highlighted by Steve Watson, Vance Johnson, and Mark Jackson (if you’re nostalgic, we can include the third amigo, Ricky Nattiel). Once an older Elway was playing with Shannon Sharpe, Rod Smith, and Ed McCaffrey, his numbers improved significantly.

So maybe he was always great but other variables outside his control limited him to just “good” for a period. Does that make him one of the top 5 greatest of all time? Top 10? I don’t know. At that level, it’s all very subjective, but I think he’s in the conversation.

My instinct, however, is that Elway’s legacy is mostly driven not by how he actually played throughout his career, but by wins and his Superbowl success. Had the Broncos not won those last two Superbowls, would he still be thought of as one of the greatest? Or would he be considered like Jim Kelly (similar numbers and win rate, but 0-4 in championships)?

Just for fun, I attempted a QB ranking on the 1983-98 data using TANY/a with adjustments for year and other factors and Elway finished 6th behind Steve Young, Joe Montana, Dan Marino, Brett Favre and Troy Aikman. While I am not claiming at all that this is a “correct” ranking, I think it certainly demonstrates that passing efficiency stats are really good at identifying talent (and really good at identifying those without).

I think if people judged QBs less by wins and more by their actual play, it would lead them to a greater truth, perhaps challenging their beliefs about what is good and bad.

---

FOOTNOTES

(1) QB fumble data was not available else a 45 yard penalty per fumble would have been included.

(2) 2009 - 2018 cumulative regular season data by team against points per drive excluding defensive scores, including offensive opponent scores (pick 6, safety fumble return) and excluding drives ended by time expiring.

(3) Drive quantities were estimated from game level data: TDs + FG attempts + Punts + Turnovers + Safeties + Turnover on Downs. Drives that end with time expired are intentionally omitted. Failed 4th down conversion attempts were not available before 1990. This should be a small unbiased error.

(4) Win rate and point production based on games played not necessarily games started.

(5) Multivariate linear regression predicting cumulative win rate from 1983 - 1992 using adjusted yardage efficiency for both offensive and defensive units by passing (adjusted for TDs, INTs) sacks, rushing (adjusted for TDs), net penalty, and special teams plays (punts, missed extra points, conversion attempts and punt, kickoff return TDs all converted to yardage equivalents using historical EPA).