Group items tagged

“Estimating the Reproducibility of Psychological Science,” from Science This paper isn’t really a study; it’s the outcome of an important movement in the field of psychology. In an effort called the Reproducibility Project, researchers at dozens of universities collaborated to replicate a hundred psychology studies that were initially conducted in 2008. They ended up replicating between a third and half of the studies. Is that result bad or good? It’s inevitable that studies won’t always be replicable—if every study could be replicated, then every researcher would be right the first time; even legitimate findings can prove fragile when you try to repeat them. All the same, the paper concludes that there is “room for improvement” in psychology, especially when it comes to “cultural practices in scientific communication.” Specifically, the authors propose that “low-power research designs combined with publication bias favoring positive results together produce a literature with upwardly biased effect sizes.” In other words, the desire for novelty drives researchers to overestimate the conclusiveness of their own work. It’s a fascinating and valuable effort to make sure that psychology moves forward in the best way possible.

It’s an intuitive claim, but also a profound one. It asserts that the benefits of bilingualism extend well beyond the realm of language, and into skills that we use in every aspect of our lives. This view is now widespread, heralded by a large community of scientists, promoted in books and magazines, and pushed by advocacy organizations.

A landmark 2015 report that cast doubt on the results of dozens of published psychology studies has exposed deep divisions in the field, serving as a reality check for many working researchers but as an affront to others who continue to insist the original research was sound. On Thursday, a group of four researchers publicly challenged the report, arguing that it was statistically flawed and, as a result, wrong.The 2015 report, called the Reproducibility Project, found that less than 40 studies in a sample of 100 psychology papers in leading journals held up when retested by an independent team. The new critique by the four researchers countered that when that team’s statistical methodology was adjusted, the rate was closer to 100 percent.

The opposite of that is the hot-hand fallacy — the belief that winning streaks, whether in basketball or coin tossing, have a tendency to continue, as if propelled by their own momentum. Both misconceptions are reflections of the brain’s wired-in rejection of the power that randomness holds over our lives. Look deep enough, we instinctively believe, and we may uncover a hidden order.

A working paper published this summer has caused a stir by proposing that a classic body of research disproving the existence of the hot hand in basketball is flawed by a subtle misperception about randomness. If the analysis is correct, the possibility remains that the hot hand is real.

Taken to extremes, seeing connections that don’t exist can be a symptom of a psychiatric condition called apophenia. In less pathological forms, the brain’s hunger for pattern gives rise to superstitions (astrology, numerology) and is a driving factor in what has been called a replication crisis in science — a growing number of papers that cannot be confirmed by other laboratories.

Q-W-E-R-T-Y. Six letters that define so much of our waking lives.

We are told that we are an irrational tangle of biases, to be nudged any which way. Does this claim stand to reason?

A culture that believes its citizens are not reliably competent thinkers will treat those citizens differently to one that respects their reflective autonomy. Which kind of culture do we want to be? And we do have a choice. Because it turns out that the modern vision of compromised rationality is more open to challenge than many of its followers accept.

Modern skepticism about rationality is largely motivated by years of experiments on cognitive bias.

It’s no secret that therapies that look promising in mice rarely work in people. But too often, experimental treatments that succeed in one mouse population do not even work in other mice, suggesting that many rodent studies may be flawed from the start.

Zimbardo’s standard narrative of the Stanford prison experiment offers the prisoners’ emotional responses as proof of how powerfully affected they were by the guards’ mistreatment. The shock of real imprisonment provides a simpler and far less groundbreaking explanation. It may also have had legal implications, should prisoners have thought to pursue them. Korpi told me that the greatest regret of his life was failing to sue Zimbardo.