JOEL BEST

What makes this statistic so bad?

But people rarely ask questions of this sort when they encounter statistics. Most of the time, most people simply accept statistics without question.

a mutant statistic

the author's article for publication did not bother to consider the implications of child victims doubling each year.

the student

the student

the student

Bad statistics live on; they take on lives of their own.

dubious data.

stir up public outrage or fear

How to Lie With Statistics.

Statistics, then, have a bad reputation.

Yet, at the same time, we need statistics; we depend upon them to summarize and clarify the nature of our complex society.

Convincing answers to such questions demand evidence, and that usually means numbers, measurements, statistics.

"prove"

Some statistics are bad, but others are pretty good, and we need statistics -- good statistics -- to talk sensibly about social problems.

We need to think critically about statistics -- at least critically enough to suspect that the number of children gunned down hasn't been doubling each year since 1950.

Innumeracy

Too few people, he argued, are comfortable with basic mathematical principles, and this makes them poor judges of the numbers they encounter

Social statistics describe society

The people who bring social statistics to our attention have reasons for doing so; they inevitably want something, just as reporters and the other media figures who repeat and publicize statistics have their own goals.

Bad statistics come from conservatives on the political right and liberals on the left, from wealthy corporations and powerful government agencies, and from advocates of the poor and the powerless.

Bad statistics come from conservatives on the political right and liberals on the left, from wealthy corporations and powerful government agencies, and from advocates of the poor and the powerless.

We need a general approach, an orientation, a mind-set that we can use to think about new statistics that we encounter. We ought to approach statistics thoughtfully.

One choice is to approach statistics critically.

The critical recognize that this is an inevitable limitation of statistics.

Moreover, they realize that every statistic is the product of choices -- the choice between defining a category broadly or narrowly, the choice of one measurement over another, the choice of a sample. People choose definitions, measurements, and samples for all sorts of reasons: Perhaps they want to emphasize some aspect of a problem; perhaps it is easier or cheaper to gather data in a particular way -- many considerations can come into play. Every statistic is a compromise among choices. This means that every definition -- and every measurement and every sample -- probably has limitations and can be criticized.

Being critical means more than simply pointing to the flaws in a statistic. Again, every statistic has flaws. The issue is whether a particular statistic's flaws are severe enough to damage its usefulness. Is the definition so broad that it encompasses too many false positives (or so narrow that it excludes too many false negatives)? How would changing the definition alter the statistic? Similarly, how do the choices of measurements and samples affect the statistic? What would happen if different measures or samples were chosen? And how is the statistic used? Is it being interpreted appropriately, or has its meaning been mangled to create a mutant statistic? Are the comparisons that are being made appropriate, or are apples being confused with oranges? How do different choices produce the conflicting numbers found in stat wars? These are the sorts of questions the critical ask.

Statistics are one of the standard types of evidence used by people in our society.

Without statistics, we limit our ability to think thoughtfully about our society; without statistics, we have no accurate ways of judging how big a problem may be, whether it is getting worse, or how well the policies designed to address that problem actually work.

The goal is not to memorize a list, but to develop a thoughtful approach.

Being critical, it seems, involves an impossible amount of work.

Rather, being critical means appreciating the inevitable limitations that affect all statistics,

Claims about social problems often feature dramatic, compelling examples; the critical might ask whether an example is likely to be a typical case or an extreme, exceptional instance. Claims about social problems often include quotations from different sources, and the critical might wonder why those sources have spoken and why they have been quoted: Do they have particular expertise? Do they stand to benefit if they influence others? Claims about social problems usually involve arguments about the problem's causes and potential solutions. The critical might ask whether these arguments are convincing. Are they logical? Does the proposed solution seem feasible and appropriate? And so on. Being critical -- adopting a skeptical, analytical stance when confronted with claims -- is an approach that goes far beyond simply dealing with statistics.

Statistics are not magical. Nor are they always true -- or always false. Nor need they be incomprehensible.

When we fail to think critically, the statistics we hear might just as well be magical.