Opinion | Bias Is a Big Problem. But So Is 'Noise.' - The New York Times - 1 views
-
The word “bias” commonly appears in conversations about mistaken judgments and unfortunate decisions. We use it when there is discrimination, for instance against women or in favor of Ivy League graduates
-
the meaning of the word is broader: A bias is any predictable error that inclines your judgment in a particular direction. For instance, we speak of bias when forecasts of sales are consistently optimistic or investment decisions overly cautious.
- ...26 more annotations...
-
when it comes to mistaken judgments and unfortunate decisions, there is another type of error that attracts far less attention: noise.
-
To see the difference between bias and noise, consider your bathroom scale. If on average the readings it gives are too high (or too low), the scale is biased
-
It is hard to escape the conclusion that sentencing is in part a lottery, because the punishment can vary by many years depending on which judge is assigned to the case and on the judge’s state of mind on that day. The judicial system is unacceptably noisy.
-
The average difference between the sentences that two randomly chosen judges gave for the same crime was more than 3.5 years. Considering that the mean sentence was seven years, that was a disconcerting amount of noise.
-
If it shows different readings when you step on it several times in quick succession, the scale is noisy.
-
How much of a difference would you expect to find between the premium values that two competent underwriters assigned to the same risk?
-
But the typical difference we found between two underwriters was an astonishing 55 percent of their average premium — more than five times as large as the executives had expected.
-
Many other studies demonstrate noise in professional judgments. Radiologists disagree on their readings of images and cardiologists on their surgery decisions
-
A company’s hiring decisions could be unbiased overall if some of its recruiters favor men and others favor women. However, its hiring decisions would be noisy, and the company would make many bad choices
-
for instance, a judge’s mood, fatigue and even the weather can all have modest but detectable effects on judicial decisions.
-
people can have different general tendencies. Judges often vary in the severity of the sentences they mete out: There are “hanging” judges and lenient ones.
-
People can have not only different general tendencies (say, whether they are harsh or lenient) but also different patterns of assessment (say, which types of cases they believe merit being harsh or lenient about).
-
Underwriters differ in their views of what is risky, and doctors in their views of which ailments require treatment.
-
Guidelines, such as those often used in medicine, can help professionals reach better and more uniform decisions
-
imposing structure and discipline in interviews and other forms of assessment tends to improve judgments of job candidates.
-
No noise-reduction techniques will be deployed, however, if we do not first recognize the existence of noise.
-
Organizations and institutions, public and private, will make better decisions if they take noise seriously.