On what’s currently an extremely sensitive topic – the causes of gun violence – the Washington Post’s Max Fisher pens what is quite possibly the most disappointing and naive article I’ve ever read on statistics. The link is here:
I’ll share the article’s main issues below, but the damage has already been done. This story made it to the headline page of yahoo.com, where 2300 people have commented – not just read, but commented on – this issue.
Issue 1: The headline: The headline states that this “Comparison Suggests That There’s no Link.” Classic introductory statistics error. Statistics can only provide evidence of a link. Thus, even if the rest of this “Comparison” was well done, all we could do claim based on this study is note there wasn’t enough evidence that an association between video games and violence exists. We can not, however, suggest no link. Big difference.
Issue 2: Sample Size: Any statistics student can tell you after week two that a sample size of 10 is probably too small to make any strong claims.
Issue 3: Sample Choice: The countries chosen for this comparison are the ones with the largest video game consumption. This seems quite arbitrary. There are about 15 other ways to choose a sample, each of which would yield a scatter plot with a different trend line and different story.
Issue 4: Outliers: Again, Intro Stats 101. China, South Korea and Netherlands are both possible outliers with regards to video game spending. As a result, regression lines – and correlation coefficients – are significantly changed due to the effect of these points. Remove South Korea and the Netherlands – and their different laws with regards to gun control – and suddenly you might have an association!
Issue 5: The graphs: Holy cow. This graph is particularly nauseating. First, a linear trend line is not the only way to identify an association. For example, a U-shape or another curved relationship is plausible in many scatter plots. Second, even if the trend was linear, for the association to be significant it would certainly not have to take the form of the graph in question. Any slight increase over x-axis creates the possibility of a positive correlation, not necessarily one of the magnitude used in this example.
Issue 6: Variable Choice: Obviously, the intent is to show the association – or lack thereof – between violent video games and gun violence. Unfortunately, the video game variable is (1) Not violent video games and (2) not measured per capita. The effect of not measuring violent video game consumption is particularly important. For example, if I wanted to know how the effect of smoking on lung cancer, its more important for me to measure this association via cigarette usage as opposed to all forms of smoking.
Max, you probably won’t read this, but if you do, please retract. Thousands of readers are now thinking to themselves “well, video games don’t cause violence,” and worse yet, there’s probably a parent or two who will let their sons or daughters continue to play violent video games. Better research has been done here, here, and here, the general conclusions of which suggest that video games are associated with increased aggressiveness and physiological desensitization to violence.