Penalty rates in the NFL

In a 2012 paper in Economic Inquiry, the University of Mississippi’s Carl Kitchens described how a repositioning of an NFL referee led to a change in the frequency of offensive holding penalties. Kitchens writes:

The results suggest that the detection effect is large. Simply by repositioning the officials in the NFL, the players with an extra set of eyes on them experienced a 20% increase in the number of called penalties, while the set of players who had the set of eyes removed had a large decrease in the number of penalties detected.

Kitchens used two full years of play-by-play data, 2009, and 2010, to reach his conclusion. His models also suggested that the change was largest on run plays, and negligible on pass plays. The paper, in its original form, can be read here, and in published form, read here.

Referee behavior has always fascinated me, and in an ongoing project, I’m exploring the distribution of NFL judgement penalties, including holding calls. As a result, I wanted to do some exploring of Kitchens’ results.

Anyways, using Brian Burke’s play-by-play data (here), I sorted all NFL plays since 2002. I excluded erroneous plays on which offensive holding couldn’t occur, or those in which I didn’t care as much if it did (any kicking play, QB kneels, spikes, etc). When I finalize all of this, I’ll make sure to post the code and data, since I think it can be a really great resource once its parsed out.

While there is plenty of work left to do, my initial results were curious.

Here’s a graph of the infraction rate for two of the calls which involve the largest amounts of discretion on behalf of officials, offensive holding (separated by play type, run or pass), and defensive pass interference (pass plays only).

As first indicated by Kitchens’ statistical models, it does indeed appear that the 2010 rule change only had an impact on holding calls on running plays, and not on passing ones.

However, I found it striking that while defensive pass interference rates have almost always held steady over time, holding calls (particularly on running plays) have been fairly volatile. For instance, there was a 40% drop in holding penalty rate on run plays after 2005 (this is a much larger difference than the increase associated with the 2009 rule change).

Further, there’s been a fairly steady increase in holding rates on run plays since 2006 – perhaps the 2009 jump was simply part of a larger pattern of increased infractions.

I’ll have more as I weed through the data. Until then, thanks for reading.