Group items tagged

“Estimating the Reproducibility of Psychological Science,” from Science This paper isn’t really a study; it’s the outcome of an important movement in the field of psychology. In an effort called the Reproducibility Project, researchers at dozens of universities collaborated to replicate a hundred psychology studies that were initially conducted in 2008. They ended up replicating between a third and half of the studies. Is that result bad or good? It’s inevitable that studies won’t always be replicable—if every study could be replicated, then every researcher would be right the first time; even legitimate findings can prove fragile when you try to repeat them. All the same, the paper concludes that there is “room for improvement” in psychology, especially when it comes to “cultural practices in scientific communication.” Specifically, the authors propose that “low-power research designs combined with publication bias favoring positive results together produce a literature with upwardly biased effect sizes.” In other words, the desire for novelty drives researchers to overestimate the conclusiveness of their own work. It’s a fascinating and valuable effort to make sure that psychology moves forward in the best way possible.

Perhaps even more potentially problematic than negative personal experience is the possibility that bias may influence research quality: its design, execution, evaluation, and interpretation. In 1975, Stephen Abramowitz and his colleagues sent a fake manuscript to eight hundred reviewers from the American Psychological Association—four hundred more liberal ones (fellows of the Society for the Psychological Study of Social Issues and editors of the Journal of Social Issues) and four hundred less liberal (social and personality psychologists who didn’t fit either of the other criteria). The paper detailed the psychological well-being of student protesters who had occupied a college administration building and compared them to their non-activist classmates. In one version, the study found that the protesters were more psychologically healthy. In another, it was the more passive group that emerged as mentally healthier. The rest of the paper was identical. And yet, the two papers were not evaluated identically. A strong favorable reaction was three times more likely when the paper echoed one’s political beliefs—that is, when the more liberal reviewers read the version that portrayed the protesters as healthier.

All these studies and analyses are classic examples of confirmation bias: when it comes to questions of subjective belief, we more easily believe the things that mesh with our general world view. When something clashes with our vision of how things should be, we look immediately for the flaws.

Henrich’s work with the ultimatum game was an example of a small but growing countertrend in the social sciences, one in which researchers look straight at the question of how deeply culture shapes human cognition. His new colleagues in the psychology department, Heine and Norenzayan, were also part of this trend. Heine focused on the different ways people in Western and Eastern cultures perceived the world, reasoned, and understood themselves in relationship to others. Norenzayan’s research focused on the ways religious belief influenced bonding and behavior. The three began to compile examples of cross-cultural research that, like Henrich’s work with the Machiguenga, challenged long-held assumptions of human psychological universality.

As Heine, Norenzayan, and Henrich furthered their search, they began to find research suggesting wide cultural differences almost everywhere they looked: in spatial reasoning, the way we infer the motivations of others, categorization, moral reasoning, the boundaries between the self and others, and other arenas. These differences, they believed, were not genetic. The distinct ways Americans and Machiguengans played the ultimatum game, for instance, wasn’t because they had differently evolved brains. Rather, Americans, without fully realizing it, were manifesting a psychological tendency shared with people in other industrialized countries that had been refined and handed down through thousands of generations in ever more complex market economies. When people are constantly doing business with strangers, it helps when they have the desire to go out of their way (with a lawsuit, a call to the Better Business Bureau, or a bad Yelp review) when they feel cheated. Because Machiguengan culture had a different history, their gut feeling about what was fair was distinctly their own. In the small-scale societies with a strong culture of gift-giving, yet another conception of fairness prevailed. There, generous financial offers were turned down because people’s minds had been shaped by a cultural norm that taught them that the acceptance of generous gifts brought burdensome obligations. Our economies hadn’t been shaped by our sense of fairness; it was the other way around.

Studies show that Western urban children grow up so closed off in man-made environments that their brains never form a deep or complex connection to the natural world. While studying children from the U.S., researchers have suggested a developmental timeline for what is called “folkbiological reasoning.” These studies posit that it is not until children are around 7 years old that they stop projecting human qualities onto animals and begin to understand that humans are one animal among many. Compared to Yucatec Maya communities in Mexico, however, Western urban children appear to be developmentally delayed in this regard. Children who grow up constantly interacting with the natural world are much less likely to anthropomorphize other living things into late childhood.

A major investigation into scores of claims made in psychology research journals has delivered a bleak verdict on the state of the science. An international team of experts repeated 100 experiments published in top psychology journals and found that they could reproduce only 36% of original findings.

It’s an intuitive claim, but also a profound one. It asserts that the benefits of bilingualism extend well beyond the realm of language, and into skills that we use in every aspect of our lives. This view is now widespread, heralded by a large community of scientists, promoted in books and magazines, and pushed by advocacy organizations.

A landmark 2015 report that cast doubt on the results of dozens of published psychology studies has exposed deep divisions in the field, serving as a reality check for many working researchers but as an affront to others who continue to insist the original research was sound. On Thursday, a group of four researchers publicly challenged the report, arguing that it was statistically flawed and, as a result, wrong.The 2015 report, called the Reproducibility Project, found that less than 40 studies in a sample of 100 psychology papers in leading journals held up when retested by an independent team. The new critique by the four researchers countered that when that team’s statistical methodology was adjusted, the rate was closer to 100 percent.

Like a number of up-and-coming researchers in his generation, Nosek was troubled by mounting evidence that science itself—through its systems of publication, funding, and advancement—had become biased toward generating a certain kind of finding: novel, attention grabbing, but ultimately unreliable. The incentives to produce positive results were so great, Nosek and others worried, that some scientists were simply locking their inconvenient data away. Related Stories Nick Stockton Science Gets Better at Being Wrong Katie M. Palmer Science Has Its Problems, But the Web Could Be the Fix Katie M. Palmer Psychology Is in Crisis Over Whether It’s in Crisis The problem even had a name: the file drawer effect.

“Just as with the physical world, where hurricanes, tornadoes, and other ‘acts of God’ just happen, the same is true of the social world,” he wrote. “Some people just do things. They assassinate world leaders, act on poorly thought out ideologies, and leave clues at the scene of the crime. Too strong a belief in the rationality of people in general, or of the world, will lead us to seek purposive explanations where none exists.”

He says conspiracy theorists rely on what he calls “errant data,” or random minutiae within a terror attack or major event that can—and maybe should—go unexplained in reality. Those pushing conspiracies, however, seize on that unexplained info and attempt to explain it in full.It is an effort to connect every dot on the map—every blade of grass on the Grassy Knoll—even if some dots have nothing to with the larger event at all

Crisis class theory is a weirdly hopeful, terribly reductionist coping mechanism, a way to explain a world that can be unjust and needlessly cruel—but wouldn’t be if the “bad guys” controlling it all were vanquished.“There is surely some psychological comfort in believing that a horrific event like a mass murder of schoolchildren never really happened at all—that it was all fake,” he writes.

Writing about your feelings, a practice long embraced by teenagers and folk singers, is now attracting attention as a path to good health. And a recent study suggests that reflecting on your emotions could help you get over a breakup. But, one of its authors says, journaling can have its downsides. Is structured self-reflection, as some suggest, a healthy tuneup for the heart and head — or can it make hurt feelings worse?

Earlier this year, two forensic psychologists—Julia Shaw, of the University of Bedfordshire, and Stephen Porter, of the University of British Columbia—upped the ante. Writing in the January issue of the journal Psychological Science, they described a method for implanting false memories, not of getting lost in childhood but of committing a crime in adolescence. They modelled their work on Loftus’s, sending questionnaires to each of their participant’s parents to gather background information. (Any past run-ins with the law would eliminate a student from the study.) Then they divided the students into two groups and told each a different kind of false story. One group was prompted to remember an emotional event, such as getting attacked by a dog. The other was prompted to remember a crime—an assault, for example—that led to an encounter with the police. At no time during the experiments were the participants allowed to communicate with their parents.

Few things are harder to study than human language. The brains of living humans can only be studied indirectly, and language, unlike vision, has no analogue in the animal world. Vision scientists can study sight in monkeys using techniques like single-neuron recording. But monkeys don’t talk. However, in an article published today in Nature, a group of researchers, including myself, detail a discovery in birdsong that may help lead to a revised understanding of an important aspect of human language development. Almost five years ago, I sent a piece of fan mail to Ofer Tchernichovski, who had just published an article showing that, in just three or four generations, songbirds raised in isolation often developed songs typical of their species. He invited me to visit his lab, a cramped space stuffed with several hundred birds residing in souped-up climate-controlled refrigerators. Dina Lipkind, at the time Tchernichovski’s post-doctoral student, explained a method she had developed for teaching zebra finches two songs. (Ordinarily, a zebra finch learns only one song in its lifetime.) She had discovered that by switching the song of a tutor bird at precisely the right moment, a juvenile bird could learn a second, new song after it had mastered the first one. Thinking about bilingualism and some puzzles I had encountered in my own lab, I suggested that Lipkind’s method could be useful in casting light on the question of how a creature—any creature—learns to put linguistic elements together.

Each scientific field must adopt its own methods of ensuring accuracy. But ultimately, this self-reflection is a key part of the scientific process. As Bishop notes, “Science has proved itself to be an incredibly powerful method.” And yet there’s always room for further advancement. “There’s never an end point,” says Bishop. “We’re always groping towards the next thing. Sometimes science does disappear down the wrong path for a bit before it corrects itself.” For Nosek, who led the re-testing of 100 psychology papers, the current focus on reproducibility is simply part of the scientific process. “Science isn’t about truth and falsity, it’s about reducing uncertainty,” he says. “Really this whole project is science on science: Researchers doing what science is supposed to do, which is be skeptical of our own process, procedure, methods, and look for ways to improve.”