It’s still relatively early in the NFL season, but signs point to this being one of the worst seasons ever for simulation & statistics based predictors of game results. In fact, having followed several websites for the last few years, this is by far the worst I can remember each one doing as far as accuracy is concerned.
Here, I summarize results through week 7.
Football Outsiders (FO) : These guys have pretty much set the standard for NFL statistical analyses, as demonstrated by their preseason almanacs, appearances across the media, and downloadable spreadsheets with all sorts of good information. In their first five years of picking games against the spread (ATS), from 2008 to 2012, FO finished with yearly success rates of 53.7%, 51.2%, 56.1%, 52.0%, and 57.8%, respectively. There’s no public access for historical picks, but I’ve followed along and these numbers are 100% trustworthy.
This Fall, however, FO premium picks are just 42-63-2 (40%) ATS. To get a sense of how unlikely these results are, if 1000 people had tossed coins to pick the same games, only about 20 of them would finish with records that poor. Bizarre.
Team Rankings: For a while, Team Rankings (TR) kept all of their picks public, with an all-time accuracy of roughly 52%. While they’ve now switched to a pay-for-picks format, their results are still public. 2013 has also not been great campaign for TR, with a current mark at 47-57-2 ATS (45%). If 1000 people were to toss coins to pick these same NFL games, about 180 of them would finish with this percentage or worse.
Also, for what its worth, TR’s “3 star or better” picks are 8-11 in 2013, after going 85-71 from 2009-2013. TR’s results are linked here.
numberFire: Like Team Rankings, numberFire (NF) appears on ESPN’s Insider page, meaning those with Insider contest (not that much $) can access their picks. NF links their results here. All time, NF picks have a 53.8% success rate, which is pretty impressive. This Fall, posted NF picks are 22-37-1 ATS (37.7%). Again, pretty poor.
To get a sense of just how different the 2013 season has been for these sites, I plotted this and past years ATS percentages for each site, including 95% confidence intervals. While ATS picks should aim for hitting a 55% cutoff to make money, with the NFL, anything noticeably greater than 50% is decent. In this regard, all three sites have historically done well.
In 2013, however, intervals for numberFire and Football Outsiders don’t even reach a 50% cutoff.
What’s been going on? I’m not sure. It could be just bad luck, or it could be a sign that these websites aren’t all they’re cracked up to be. Here’s hoping to a speedy recovery for all three pick sites in 2013.
Other sites: There are a few other sites worth mentioning.
Sports Insights appears to be doing OK, but as I’ve written here before, this website appears keen on misleading its readers. For example, a huge pet peeve of mine is when websites use fake names for their picks. After doing this – assuming half of the named picks will do better than average – it’s easy to only highlight the ones that do the best. Sports Insights uses “Steam”, “Smart Money”, “Square” and “Best Bets” as names for their picks. Whenever I see these types of picks, I roll my eyes.
My friend Greg at Stats in the Wild has had a tough go of it too, currently at 16 games under 0.500. However, his picks are free and posted weekly, which you can’t say of most statistical simulators. As a result, the good news here is that if you don’t like Greg, it appears possible to pick against him and make a good amount of $. In 2012, Greg finished at a 54.6% clip.
Prediction Machine claims to be at 54.4% ATS this year, which would be quite good. Unfortunately, unlike many of the aforementioned websites, there doesn’t appear to be a master list of games picked. As a result, there’s no way of knowing if this number is accurate, unless you have access and the time to tabulate their picks each week. This information seems pretty critical to the judgement of each website’s success.
So, to summarize, if you’ve used statistics websites for your NFL picks (especially the three graphed), you’ve probably had an unsuccessful Fall…unless, of course, you’ve been picking against them!
Postscript: If anyone has other sites, please send them along!
StatsbyLopez 
In my defense, I was 143-119-1 against the spread last year. That’s 54.56%. I wonder why everyone is losing this year.
Reblogged this on Stats in the Wild and commented:
It’s true. I am doing terrible in the NFL this year. But so is basically everyone else.
Time to analyze the incorrect picks and search for an underlying theme. By your own admission, there is a 2% probability that this finding is due to chance. So, we have statistical significance! Something must explain this effect!
What everyone seems to be missing is: almost every year the NFL changes the rules, so every year can be a new “sport” in terms of the factors impacting the sport, and the coaches ramping up new learning curves to match. This year we have a bunch of pick-6s partially due in part to higher passing stats and lowered rushing attempts. The new rule on using the crown of the helmet is impacting the rush game, pushing more pass attempts and thus more variability. If the NFL quits messing with the rules of the game, we’ll get to a new Nash-like saddle-point in the betting market where the numerics guys will assert themselves again. BTW, when you look back on the years that the pros do poorly, the public tends to do well, and we then call that year “a square year” as the homers and favorites (correlated to about .7, as I recall) tend to come in. Those years are typically after the NFL has messed with something rule wise. If you want to look at a market where this does not happen, go look at college football where the rules hardly ever change.