The Chronicle: 5/4/2001: Telling the Truth About Damned Lies and Statistics - 0 views
-
Damned Lies and Statistics
-
Terry Elliott on 01 Jan 09According to William Faulkner, "Facts and truth really don't have much to do with each other." Could the same be held about stats and truth?
-
-
it may be the worst -- that is, the most inaccurate -- social statistic ever.
-
If anyone spots a more inaccurate social statistic, I'd love to hear
- ...37 more annotations...
-
JOEL BEST
-
What makes this statistic so bad?
-
Here is a great blog on the issue of "bad statistical behavior". http://blogs.wsj.com/numbersguy/
-
-
But people rarely ask questions of this sort when they encounter statistics. Most of the time, most people simply accept statistics without question.
-
a mutant statistic
-
the author's article for publication did not bother to consider the implications of child victims doubling each year.
-
the student
-
Bad statistics live on; they take on lives of their own.
-
dubious data.
-
stir up public outrage or fear
-
How to Lie With Statistics.
-
Statistics, then, have a bad reputation.
-
Yet, at the same time, we need statistics; we depend upon them to summarize and clarify the nature of our complex society.
-
Convincing answers to such questions demand evidence, and that usually means numbers, measurements, statistics.
-
"prove"
-
Some statistics are bad, but others are pretty good, and we need statistics -- good statistics -- to talk sensibly about social problems.
-
N, Taleb has some interesting things to say about the ability to write/speak sensibly about the probablistic world with statistics. Before the discovery of black swans in Australia it was statistically certain (p=1) that all swans are white. All it took was the observation that there is one black swan to nullify that probability.
-
-
We need to think critically about statistics -- at least critically enough to suspect that the number of children gunned down hasn't been doubling each year since 1950.
-
Innumeracy
-
Too few people, he argued, are comfortable with basic mathematical principles, and this makes them poor judges of the numbers they encounter
-
Social statistics describe society
-
The people who bring social statistics to our attention have reasons for doing so; they inevitably want something, just as reporters and the other media figures who repeat and publicize statistics have their own goals.
-
Bad statistics come from conservatives on the political right and liberals on the left, from wealthy corporations and powerful government agencies, and from advocates of the poor and the powerless.
-
Bad statistics come from conservatives on the political right and liberals on the left, from wealthy corporations and powerful government agencies, and from advocates of the poor and the powerless.
-
We need a general approach, an orientation, a mind-set that we can use to think about new statistics that we encounter. We ought to approach statistics thoughtfully.
-
One choice is to approach statistics critically.
-
The critical recognize that this is an inevitable limitation of statistics.
-
Moreover, they realize that every statistic is the product of choices -- the choice between defining a category broadly or narrowly, the choice of one measurement over another, the choice of a sample. People choose definitions, measurements, and samples for all sorts of reasons: Perhaps they want to emphasize some aspect of a problem; perhaps it is easier or cheaper to gather data in a particular way -- many considerations can come into play. Every statistic is a compromise among choices. This means that every definition -- and every measurement and every sample -- probably has limitations and can be criticized.
-
Being critical means more than simply pointing to the flaws in a statistic. Again, every statistic has flaws. The issue is whether a particular statistic's flaws are severe enough to damage its usefulness. Is the definition so broad that it encompasses too many false positives (or so narrow that it excludes too many false negatives)? How would changing the definition alter the statistic? Similarly, how do the choices of measurements and samples affect the statistic? What would happen if different measures or samples were chosen? And how is the statistic used? Is it being interpreted appropriately, or has its meaning been mangled to create a mutant statistic? Are the comparisons that are being made appropriate, or are apples being confused with oranges? How do different choices produce the conflicting numbers found in stat wars? These are the sorts of questions the critical ask.
-
Statistics are one of the standard types of evidence used by people in our society.
-
Are stats just another rhetorical tool then? I don't think that they are, but they can be used that way. And I don't mean rherorical in a pejorative sense here. Language and statistics are by there very existence a part of a sociable world. As such they are part of every persuasive toolbox that begins with passion and moves from that toward politics. We all want what we want. Statistics is part of the language that helps us get that. THAT is what we need to be critical about. Are we being fair in their use?
-
-
Without statistics, we limit our ability to think thoughtfully about our society; without statistics, we have no accurate ways of judging how big a problem may be, whether it is getting worse, or how well the policies designed to address that problem actually work.
-
The goal is not to memorize a list, but to develop a thoughtful approach.
-
What Best is arguing for is the internalization of statistical habits, of a statistical stance. But what makes this potential stance any more worth adopting than a meditative approach or an empathic approach or a kinesthetic approach or even a punk approach. Literary criticism comes in for a world of legitimate criticism, but one of its strengths is in recognizing that the ability to adopt multiple critical stances allows us to be like the blind men who are trying to identify the elephant. Statistics is just one more hand to make the task more enlightened.
-
-
Being critical, it seems, involves an impossible amount of work.
-
Rather, being critical means appreciating the inevitable limitations that affect all statistics,
-
Claims about social problems often feature dramatic, compelling examples; the critical might ask whether an example is likely to be a typical case or an extreme, exceptional instance. Claims about social problems often include quotations from different sources, and the critical might wonder why those sources have spoken and why they have been quoted: Do they have particular expertise? Do they stand to benefit if they influence others? Claims about social problems usually involve arguments about the problem's causes and potential solutions. The critical might ask whether these arguments are convincing. Are they logical? Does the proposed solution seem feasible and appropriate? And so on. Being critical -- adopting a skeptical, analytical stance when confronted with claims -- is an approach that goes far beyond simply dealing with statistics.
-
Statistics are not magical. Nor are they always true -- or always false. Nor need they be incomprehensible.
-
When we fail to think critically, the statistics we hear might just as well be magical.