The NFL draft – where we stand in 2016

Another NFL draft has come and gone, and with it has come the predictable displays of unyielding optimism, stale and arguably race-based generalizations of player skill, and, as a relative newcomer in 2016, lazy misuse of the term analytics.

In following along this spring, it became clear that what is mainstream knowledge among researchers is far from it in the national media. This despite a decent amount of both academic and non-academic research into the topic.

For those new to the scene, or even for a few veterans who may have missed an article or two along the way, I decided to write a quick review of what’s out there. Note that many of the following points are related to one another.

1. Top draft picks are overvalued. 

In an efficient market, the value of picks traded between teams would be equivalent.  That is, if Pick X was traded for Picks Y and Z, the value added by picking in spot X should equal that of picking in spots Y and Z. Interestingly, in a pair of well-regarded papers from researchers Cade Massey and Richard Thaler, the authors identified marked inefficiencies in how NFL franchises value different draft picks.

Turns out, the preferable solution is almost always to acquire more picks. In our example, better to have Y and Z than X alone. It’s why the the common suggestion among many who have studied the draft is that because teams overvalue earlier picks, it is generally a good idea to trade down. The edge goes to those who take advantage of the overconfidence shown by others.

Related: go Browns.

2. Top picks tend to be (relatively) overpaid.

In addition to being overvalued on behalf of league officials, players picked at the top of the draft have also tended to offer less of a return on their investment than those picked at the end of round one. To be specific, the NFL rookie pay scale is nearly fixed, and players picked near the top of the draft get paid substantially more. As a result, for several years, it was optimal to pick late in the first or early in the second rounds, where the expectation was to obtain a decent player with a substantially more palatable contract. See Figure III in Massey & Thaler’s paper here, as one example. The author’s aptly call this a loser’s curse; bad NFL teams are stuck picking at the top of the first round, where the expected return is actually lower.

Note that given the new CBA which updated rookie contracts in 2014, it is less clear if such a surplus value still exists in today’s NFL.

3. Teams are, by and large, making guesses

One of the most consistent findings in current literature is that a team’s prowess in picking good players doesn’t translate from one season to the next. Chase reached such a conclusion a few years ago, and I really liked Neil’s scatter plot below, which shows the lack of a year-to-year consistency in team-level returns over expectation.

 

To sum, despite all that you’ll hear to the contrary, there’s no evidence to date that an organizational-level ability to identify talent in the draft is repeatable.

And if team’s are unable to consistently identify good picks in the draft, it’s safe to say that draft pundits can’t, either. In fact, when I compared Mel Kiper’s draft grades to his re-grade of that same draft after a few seasons of play, the link was wink. That is, Kiper’s post-hoc analysis of a draft barely drew any of the same conclusions as his first one.

So, whenever you hear team officials boasting about an incoming player – or even when looking at the so-called draft ‘grades’ put out everywhere – it’s not much different than listening to someone saying they like heads in a coin flip or red on the roulette table.

4. Sunk costs make analysis difficult.

Let’s take an interlude to talk fantasy football. Imagine that you drafted Peyton Manning early in 2016, making a substantial investment in a quarterback who you anticipated would lead your squad to greener pastures. Turns out, Manning stunk from week 1 onwards. But there he was in your week 5 line-up, then again in your week 10 line-up, and so on. You couldn’t move on!

Turns out, NFL organizations also suffer from the same problem. Such was the conclusion of Quinn Keifer, who found that players chosen at the end of Rd. 1 received substantially more playing time than those who were picked at the beginning of Rd. 2. In other words, the premium made on first round picks was responsible for eventual increases in games started. Keifer blames sunk costs, in which prior investments have led to teams making irrational decisions down the road. This is also what ruined any fantasy player who stuck with Manning.

This result matters when it comes to analyzing player success. If organizations are afraid to cut loose players that they drafted early, it would muddle any link between draft round and player success, and it could artificially inflate the link between early draft picks and long term success.

5. What else is out there?

I explored a few major findings above, but here is a brief synopsis of other interesting material.

-There may be position-level variability as far as identifying talent. Of course, this becomes complicated at some positions, such as linebacker. Write Zach & Rob, “It’s also possible that players with good pedigrees and name recognition are being given preferential treatment when awards are granted, in the absence of meaningful performance data (like what exists for offensive skill players) to challenge our perceptions.”

-Neil and Alison look at position-level value here.

-Sports Analytics groups out of Carnegie Mellon and Harvard have done some solid work.

-There are oodles of NFL draft value charts. Examples include the original one credited to Jimmy Johnson, newer ones from Chase Stuart and the Harvard cohort, and perhaps the most methodologically driven edition from Michael Schuckers.

-In a more recent article, a pair of authors conclude that picks made from teams that trade up tend to outperform picks made when teams stand pat, as judged by games started and approximate value in a player’s first three years. However, my initial reaction to this work is one of skepticism, in large part because of what we went over earlier about sunk costs. If teams trade up for a player, that investment could lend itself towards a team aggressively showcasing him. Additionally, the author’s do not mention the often prohibitive cost of trading up, which could offset any gains in surplus drafting.

-In the next few days, I’ll have some posts that look at the history of the draft and team success. More to come.

So what did we learn?

It is extremely difficult to consistently identify talent in the NFL draft. However, there are inefficiencies, which generally arise when teams overvalue players by trading up to acquire them.  Such displays of confidence belie a process which is, by and large, no different than picking heads or tails before a coin toss.

Perhaps Massey said it best in an article on Vox:

“It’s basically a coin flip, but teams are paying a great deal for the right to call which side of the coin.”