THE TROUBLE WITH CRIME STATISTICS It’s surprisingly hard to say what makes crime go up or down.

By Matthew Hutson, January 9, 2020 The New Yorker. Available online

This article is well worth reading because it deals with many measurement and analysis issues that evaluators should be very familiar with.

Selected quotes follow:

Criminologists sometimes describe crime as a “chaotic system,” and countless factors contribute to it.

The first problem with understanding crime is that measuring it is harder than it sounds. 

Even if we had perfect measurements of crime rates, we’d need to sort correlation from causation. 

The passage of time makes it especially difficult to sort correlation from causation. Statisticians are always on the lookout for a phenomenon they call “regression toward the mean


Mechanisms matter, too: in addition to knowing that something works, we want to know why it works.

In some cases, decades must pass before the effects of an intervention become visible

In some cases, findings reverse, then reverse again. 

Beyond measuring crime and determining its causes, a third difficulty lies in predicting the effects of interventions. Peter Grabosky, a political scientist, wrote that “the tendency to overgeneralize” might be “the most common pitfall” in the study of anti-crime interventions: “

The converse is also true: crime isn’t a purely local phenomenon, and interventions in one place may affect criminal behavior in another.

Sampson thinks that criminologists should spend less time trying to figure out what causes crime—in many cases, it’s an impossible task—and turn, instead, to investigating the effects of law-enforcement policies

Criminologists may disagree on questions of causality, but they agree that outsiders underestimate the complexity of criminology. “Everyone thinks they know what causes crime,” Sampson told me. Lauritsen concurred: “Everybody has a strong view that some factor is responsible, whether it’s video games, bad music, or sexist attitudes,” she said. Kleiman complained about the “very primitive models people have in their heads” when it comes to crime: “Most of those models imply that more severity of punishment is better, which is almost certainly false.” He went on, “Anyone who hasn’t studied this professionally has more confidence than they ought to have. You have to really look at it hard to see how confusing it is.”

Criminologists face a problem that’s common in many fields: overdetermination. Why does someone commit a crime? Was it peer pressure, poverty, a broken family, broken windows, bad genes, bad parenting, under-policing, leaded gasoline, Judas Priest? “You could just keep stepping back and back and back and back, and you wonder when, ultimately, you’re going to draw a line,” Lauritsen told me. “It could be drawn at probably thousands of points.” Criminologists aren’t the only researchers who study overdetermined subjects: biologists, who have long sought specific genes for diseases, have come to realize that many traits and illnesses may be “omnigenic”—determined by countless genes. The sociologist David Matza summed up the difficulty, in 1964: “When factors become too numerous, we are in the hopeless position of arguing that everything matters.

Still, it’s human nature to prefer comprehensible stories to uninterpretable complexity. 

Perhaps, whenever someone offers up an especially compelling explanation for a rise or fall in crime, we should be wary. We might recognize that criminology—a field with direct bearing on many charged issues—is also slow and confounding, with answers that may come decades late or not at all. The moral and social complexity of crime makes simple accounts of it all the more appealing. In hearing an explanation for its rise or fall, we might ask: What kind of story is its bearer trying to tell?

Known unknowns: How to communicate certainly in an uncertain world

“From the speed of global warming to the likelihood of developing cancer, we must grasp uncertainty to understand the world. Here’s how to know your unknowns” By Anne Marthe van der Bles, New Scientist, 3rd July 2019

https://www.newscientist.com/article/mg24332372-600-known-unknowns-how-to-communicate-certainly-in-an-uncertain-world/

The above reminds me of the philosophers’ demands in The Hitchhikers Guide to the Galaxy: “We demand rigidly defined areas of doubt and uncertainty!” The philosophers were representatives of the Amalgamated Union of Philosophers, Sages, Luminaries and Other Thinking Persons and they wanted the universe’s second-best computer (Deep Thought) turned off, because of a demarcation dispute/.  It turns out, according to the above paper, that their demands were not so unreasonable after all :-)

The surprising usefulness of simple measures: HappyOrNot terminals

as described in this very readable article by David Owen:
Customer Satisfaction at the Push of  a Button – HappyOrNot terminals look simple, but the information they gather is revelatory. New Yorker, 2 February 2018, pages 26-29

Read the full article here

Points of interest covered by the article include:

  1. What is so good about them
  2. Why they work so well
  3. Can people “game” the data that is collected
  4. The value of immediacy of data collection
  5. How value is added to data points by information about location and time
  6. Example of real life large scale applications
  7. What is the worse thing that could happen

Other articles on the same subject:

Rick Davies comment: I like the design of the simple experiment described in the first para of this article. Because the locations of the petrol stations were different, and thus not comparable, the managers swapped the “treatment” given to each station i.e the staff they thought were making a difference to the performance of these stations.

%d bloggers like this: