Skip to main content

Home/ TOK Friends/ Group items tagged meta-research

Rss Feed Group items tagged

Javier E

Instagram's Algorithm Delivers Toxic Video Mix to Adults Who Follow Children - WSJ - 0 views

  • Instagram’s Reels video service is designed to show users streams of short videos on topics the system decides will interest them, such as sports, fashion or humor. 
  • The Meta Platforms META -1.04%decrease; red down pointing triangle-owned social app does the same thing for users its algorithm decides might have a prurient interest in children, testing by The Wall Street Journal showed.
  • The Journal sought to determine what Instagram’s Reels algorithm would recommend to test accounts set up to follow only young gymnasts, cheerleaders and other teen and preteen influencers active on the platform.
  • ...30 more annotations...
  • “Our systems are effective at reducing harmful content, and we’ve invested billions in safety, security and brand suitability solutions,” said Samantha Stetson, a Meta vice president who handles relations with the advertising industry. She said the prevalence of inappropriate content on Instagram is low, and that the company invests heavily in reducing it.
  • The Journal set up the test accounts after observing that the thousands of followers of such young people’s accounts often include large numbers of adult men, and that many of the accounts who followed those children also had demonstrated interest in sex content related to both children and adults
  • The Journal also tested what the algorithm would recommend after its accounts followed some of those users as well, which produced more-disturbing content interspersed with ads.
  • The Canadian Centre for Child Protection, a child-protection group, separately ran similar tests on its own, with similar results.
  • Meta said the Journal’s tests produced a manufactured experience that doesn’t represent what billions of users see. The company declined to comment on why the algorithms compiled streams of separate videos showing children, sex and advertisements, but a spokesman said that in October it introduced new brand safety tools that give advertisers greater control over where their ads appear, and that Instagram either removes or reduces the prominence of four million videos suspected of violating its standards each month. 
  • The Journal reported in June that algorithms run by Meta, which owns both Facebook and Instagram, connect large communities of users interested in pedophilic content. The Meta spokesman said a task force set up after the Journal’s article has expanded its automated systems for detecting users who behave suspiciously, taking down tens of thousands of such accounts each month. The company also is participating in a new industry coalition to share signs of potential child exploitation.
  • Following what it described as Meta’s unsatisfactory response to its complaints, Match began canceling Meta advertising for some of its apps, such as Tinder, in October. It has since halted all Reels advertising and stopped promoting its major brands on any of Meta’s platforms. “We have no desire to pay Meta to market our brands to predators or place our ads anywhere near this content,” said Match spokeswoman Justine Sacco.
  • Even before the 2020 launch of Reels, Meta employees understood that the product posed safety concerns, according to former employees.
  • Robbie McKay, a spokesman for Bumble, said it “would never intentionally advertise adjacent to inappropriate content,” and that the company is suspending its ads across Meta’s platforms.
  • Meta created Reels to compete with TikTok, the video-sharing platform owned by Beijing-based ByteDance. Both products feed users a nonstop succession of videos posted by others, and make money by inserting ads among them. Both companies’ algorithms show to a user videos the platforms calculate are most likely to keep that user engaged, based on his or her past viewing behavior
  • The Journal reporters set up the Instagram test accounts as adults on newly purchased devices and followed the gymnasts, cheerleaders and other young influencers. The tests showed that following only the young girls triggered Instagram to begin serving videos from accounts promoting adult sex content alongside ads for major consumer brands, such as one for Walmart that ran after a video of a woman exposing her crotch. 
  • When the test accounts then followed some users who followed those same young people’s accounts, they yielded even more disturbing recommendations. The platform served a mix of adult pornography and child-sexualizing material, such as a video of a clothed girl caressing her torso and another of a child pantomiming a sex act.
  • Experts on algorithmic recommendation systems said the Journal’s tests showed that while gymnastics might appear to be an innocuous topic, Meta’s behavioral tracking has discerned that some Instagram users following preteen girls will want to engage with videos sexualizing children, and then directs such content toward them.
  • Current and former Meta employees said in interviews that the tendency of Instagram algorithms to aggregate child sexualization content from across its platform was known internally to be a problem. Once Instagram pigeonholes a user as interested in any particular subject matter, they said, its recommendation systems are trained to push more related content to them.
  • Preventing the system from pushing noxious content to users interested in it, they said, requires significant changes to the recommendation algorithms that also drive engagement for normal users. Company documents reviewed by the Journal show that the company’s safety staffers are broadly barred from making changes to the platform that might reduce daily active users by any measurable amount.
  • The test accounts showed that advertisements were regularly added to the problematic Reels streams. Ads encouraging users to visit Disneyland for the holidays ran next to a video of an adult acting out having sex with her father, and another of a young woman in lingerie with fake blood dripping from her mouth. An ad for Hims ran shortly after a video depicting an apparently anguished woman in a sexual situation along with a link to what was described as “the full video.”
  • Instagram’s system served jarring doses of salacious content to those test accounts, including risqué footage of children as well as overtly sexual adult videos—and ads for some of the biggest U.S. brands.
  • Part of the problem is that automated enforcement systems have a harder time parsing video content than text or still images. Another difficulty arises from how Reels works: Rather than showing content shared by users’ friends, the way other parts of Instagram and Facebook often do, Reels promotes videos from sources they don’t follow
  • In an analysis conducted shortly before the introduction of Reels, Meta’s safety staff flagged the risk that the product would chain together videos of children and inappropriate content, according to two former staffers. Vaishnavi J, Meta’s former head of youth policy, described the safety review’s recommendation as: “Either we ramp up our content detection capabilities, or we don’t recommend any minor content,” meaning any videos of children.
  • At the time, TikTok was growing rapidly, drawing the attention of Instagram’s young users and the advertisers targeting them. Meta didn’t adopt either of the safety analysis’s recommendations at that time, according to J.
  • Stetson, Meta’s liaison with digital-ad buyers, disputed that Meta had neglected child safety concerns ahead of the product’s launch. “We tested Reels for nearly a year before releasing it widely, with a robust set of safety controls and measures,” she said. 
  • After initially struggling to maximize the revenue potential of its Reels product, Meta has improved how its algorithms recommend content and personalize video streams for users
  • Among the ads that appeared regularly in the Journal’s test accounts were those for “dating” apps and livestreaming platforms featuring adult nudity, massage parlors offering “happy endings” and artificial-intelligence chatbots built for cybersex. Meta’s rules are supposed to prohibit such ads.
  • The Journal informed Meta in August about the results of its testing. In the months since then, tests by both the Journal and the Canadian Centre for Child Protection show that the platform continued to serve up a series of videos featuring young children, adult content and apparent promotions for child sex material hosted elsewhere. 
  • As of mid-November, the center said Instagram is continuing to steadily recommend what the nonprofit described as “adults and children doing sexual posing.”
  • Meta hasn’t offered a timetable for resolving the problem or explained how in the future it would restrict the promotion of inappropriate content featuring children. 
  • The Journal’s test accounts found that the problem even affected Meta-related brands. Ads for the company’s WhatsApp encrypted chat service and Meta’s Ray-Ban Stories glasses appeared next to adult pornography. An ad for Lean In Girls, the young women’s empowerment nonprofit run by former Meta Chief Operating Officer Sheryl Sandberg, ran directly before a promotion for an adult sex-content creator who often appears in schoolgirl attire. Sandberg declined to comment. 
  • Through its own tests, the Canadian Centre for Child Protection concluded that Instagram was regularly serving videos and pictures of clothed children who also appear in the National Center for Missing and Exploited Children’s digital database of images and videos confirmed to be child abuse sexual material. The group said child abusers often use the images of the girls to advertise illegal content for sale in dark-web forums.
  • The nature of the content—sexualizing children without generally showing nudity—reflects the way that social media has changed online child sexual abuse, said Lianna McDonald, executive director for the Canadian center. The group has raised concerns about the ability of Meta’s algorithms to essentially recruit new members of online communities devoted to child sexual abuse, where links to illicit content in more private forums proliferate.
  • “Time and time again, we’ve seen recommendation algorithms drive users to discover and then spiral inside of these online child exploitation communities,” McDonald said, calling it disturbing that ads from major companies were subsidizing that process.
Javier E

His Job Was to Make Instagram Safe for Teens. His 14-Year-Old Showed Him What the App W... - 0 views

  • The experience of young users on Meta’s Instagram—where Bejar had spent the previous two years working as a consultant—was especially acute. In a subsequent email to Instagram head Adam Mosseri, one statistic stood out: One in eight users under the age of 16 said they had experienced unwanted sexual advances on the platform over the previous seven days.
  • For Bejar, that finding was hardly a surprise. His daughter and her friends had been receiving unsolicited penis pictures and other forms of harassment on the platform since the age of 14, he wrote, and Meta’s systems generally ignored their reports—or responded by saying that the harassment didn’t violate platform rules.
  • “I asked her why boys keep doing that,” Bejar wrote to Zuckerberg and his top lieutenants. “She said if the only thing that happens is they get blocked, why wouldn’t they?”
  • ...39 more annotations...
  • For the well-being of its users, Bejar argued, Meta needed to change course, focusing less on a flawed system of rules-based policing and more on addressing such bad experiences
  • The company would need to collect data on what upset users and then work to combat the source of it, nudging those who made others uncomfortable to improve their behavior and isolating communities of users who deliberately sought to harm others.
  • “I am appealing to you because I believe that working this way will require a culture shift,” Bejar wrote to Zuckerberg—the company would have to acknowledge that its existing approach to governing Facebook and Instagram wasn’t working.
  • During and after Bejar’s time as a consultant, Meta spokesman Andy Stone said, the company has rolled out several product features meant to address some of the Well-Being Team’s findings. Those features include warnings to users before they post comments that Meta’s automated systems flag as potentially offensive, and reminders to be kind when sending direct messages to users like content creators who receive a large volume of messages. 
  • Meta’s classifiers were reliable enough to remove only a low single-digit percentage of hate speech with any degree of precision.
  • Bejar was floored—all the more so when he learned that virtually all of his daughter’s friends had been subjected to similar harassment. “DTF?” a user they’d never met would ask, using shorthand for a vulgar proposition. Instagram acted so rarely on reports of such behavior that the girls no longer bothered reporting them. 
  • Meta’s own statistics suggested that big problems didn’t exist. 
  • Meta had come to approach governing user behavior as an overwhelmingly automated process. Engineers would compile data sets of unacceptable content—things like terrorism, pornography, bullying or “excessive gore”—and then train machine-learning models to screen future content for similar material.
  • While users could still flag things that upset them, Meta shifted resources away from reviewing them. To discourage users from filing reports, internal documents from 2019 show, Meta added steps to the reporting process. Meta said the changes were meant to discourage frivolous reports and educate users about platform rules. 
  • The outperformance of Meta’s automated enforcement relied on what Bejar considered two sleights of hand. The systems didn’t catch anywhere near the majority of banned content—only the majority of what the company ultimately removed
  • “Please don’t talk about my underage tits,” Bejar’s daughter shot back before reporting his comment to Instagram. A few days later, the platform got back to her: The insult didn’t violate its community guidelines.
  • Also buttressing Meta’s statistics were rules written narrowly enough to ban only unambiguously vile material. Meta’s rules didn’t clearly prohibit adults from flooding the comments section on a teenager’s posts with kiss emojis or posting pictures of kids in their underwear, inviting their followers to “see more” in a private Facebook Messenger group. 
  • “Mark personally values freedom of expression first and foremost and would say this is a feature and not a bug,” Rosen responded
  • Narrow rules and unreliable automated enforcement systems left a lot of room for bad behavior—but they made the company’s child-safety statistics look pretty good according to Meta’s metric of choice: prevalence.
  • Defined as the percentage of content viewed worldwide that explicitly violates a Meta rule, prevalence was the company’s preferred measuring stick for the problems users experienced.
  • According to prevalence, child exploitation was so rare on the platform that it couldn’t be reliably estimated, less than 0.05%, the threshold for functional measurement. Content deemed to encourage self-harm, such as eating disorders, was just as minimal, and rule violations for bullying and harassment occurred in just eight of 10,000 views. 
  • “There’s a grading-your-own-homework problem,”
  • Meta defines what constitutes harmful content, so it shapes the discussion of how successful it is at dealing with it.”
  • It could reconsider its AI-generated “beauty filters,” which internal research suggested made both the people who used them and those who viewed the images more self-critical
  • the team built a new questionnaire called BEEF, short for “Bad Emotional Experience Feedback.
  • A recurring survey of issues 238,000 users had experienced over the past seven days, the effort identified problems with prevalence from the start: Users were 100 times more likely to tell Instagram they’d witnessed bullying in the last week than Meta’s bullying-prevalence statistics indicated they should.
  • “People feel like they’re having a bad experience or they don’t,” one presentation on BEEF noted. “Their perception isn’t constrained by policy.
  • they seemed particularly common among teens on Instagram.
  • Among users under the age of 16, 26% recalled having a bad experience in the last week due to witnessing hostility against someone based on their race, religion or identity
  • More than a fifth felt worse about themselves after viewing others’ posts, and 13% had experienced unwanted sexual advances in the past seven days. 
  • The vast gap between the low prevalence of content deemed problematic in the company’s own statistics and what users told the company they experienced suggested that Meta’s definitions were off, Bejar argued
  • To minimize content that teenagers told researchers made them feel bad about themselves, Instagram could cap how much beauty- and fashion-influencer content users saw.
  • Proving to Meta’s leadership that the company’s prevalence metrics were missing the point was going to require data the company didn’t have. So Bejar and a group of staffers from the Well-Being Team started collecting it
  • And it could build ways for users to report unwanted contacts, the first step to figuring out how to discourage them.
  • One experiment run in response to BEEF data showed that when users were notified that their comment or post had upset people who saw it, they often deleted it of their own accord. “Even if you don’t mandate behaviors,” said Krieger, “you can at least send signals about what behaviors aren’t welcome.”
  • But among the ranks of Meta’s senior middle management, Bejar and Krieger said, BEEF hit a wall. Managers who had made their careers on incrementally improving prevalence statistics weren’t receptive to the suggestion that the approach wasn’t working. 
  • After three decades in Silicon Valley, he understood that members of the company’s C-Suite might not appreciate a damning appraisal of the safety risks young users faced from its product—especially one citing the company’s own data. 
  • “This was the email that my entire career in tech trained me not to send,” he says. “But a part of me was still hoping they just didn’t know.”
  • “Policy enforcement is analogous to the police,” he wrote in the email Oct. 5, 2021—arguing that it’s essential to respond to crime, but that it’s not what makes a community safe. Meta had an opportunity to do right by its users and take on a problem that Bejar believed was almost certainly industrywide.
  • fter Haugen’s airing of internal research, Meta had cracked down on the distribution of anything that would, if leaked, cause further reputational damage. With executives privately asserting that the company’s research division harbored a fifth column of detractors, Meta was formalizing a raft of new rules for employees’ internal communication.
  • Among the mandates for achieving “Narrative Excellence,” as the company called it, was to keep research data tight and never assert a moral or legal duty to fix a problem.
  • “I had to write about it as a hypothetical,” Bejar said. Rather than acknowledging that Instagram’s survey data showed that teens regularly faced unwanted sexual advances, the memo merely suggested how Instagram might help teens if they faced such a problem.
  • The hope that the team’s work would continue didn’t last. The company stopped conducting the specific survey behind BEEF, then laid off most everyone who’d worked on it as part of what Zuckerberg called Meta’s “year of efficiency.
  • If Meta was to change, Bejar told the Journal, the effort would have to come from the outside. He began consulting with a coalition of state attorneys general who filed suit against the company late last month, alleging that the company had built its products to maximize engagement at the expense of young users’ physical and mental health. Bejar also got in touch with members of Congress about where he believes the company’s user-safety efforts fell short. 
Javier E

Psychiatry's New Guide Falls Short, Experts Say - NYTimes.com - 1 views

  • his goal was to reshape the direction of psychiatric research to focus on biology, genetics and neuroscience so that scientists can define disorders by their causes, rather than their symptoms.
  • While the Diagnostic and Statistical Manual of Mental Disorders, or D.S.M., is the best tool now available for clinicians treating patients and should not be tossed out, he said, it does not reflect the complexity of many disorders, and its way of categorizing mental illnesses should not guide research.
  • senior figures in psychiatry who have challenged not only decisions about specific diagnoses but the scientific basis of the entire enterprise. Basic research into the biology of mental disorders and treatment has stalled, they say, confounded by the labyrinth of the brain.
  • ...5 more annotations...
  • The creators of the D.S.M. in the 1960s and ’70s “were real heroes at the time,” said Dr. Steven E. Hyman, a psychiatrist and neuroscientist at the Broad Institute and a former director at the National Institute of Mental Health. “They chose a model in which all psychiatric illnesses were represented as categories discontinuous with ‘normal.’ But this is totally wrong in a way they couldn’t have imagined. So in fact what they produced was an absolute scientific nightmare. Many people who get one diagnosis get five diagnoses, but they don’t have five diseases — they have one underlying condition.”
  • Decades of spending on neuroscience have taught scientists mostly what they do not know, undermining some of their most elemental assumptions. Genetic glitches that appear to increase the risk of schizophrenia in one person may predispose others to autism-like symptoms, or bipolar disorder. The mechanisms of the field’s most commonly used drugs — antidepressants like Prozac, and antipsychosis medications like Zyprexa — have revealed nothing about the causes of those disorders. And major drugmakers have scaled back psychiatric drug development, having virtually no new biological “targets” to shoot for.
  • Dr. Insel is one of a growing number of scientists who think that the field needs an entirely new paradigm for understanding mental disorders, though neither he nor anyone else knows exactly what it will look like.
  • Dr. Hyman, Dr. Insel and other experts said they hoped that the science of psychiatry would follow the direction of cancer research, which is moving from classifying tumors by where they occur in the body to characterizing them by their genetic and molecular signatures.
  • Dr. Insel said in the interview that his motivation was not to disparage the D.S.M. as a clinical tool, but to encourage researchers and especially outside reviewers who screen proposals for financing from his agency to disregard its categories and investigate the biological underpinnings of disorders instead.
peterconnelly

Meta Will Give Researchers More Information on Political Ad Targeting - The New York Times - 0 views

  • Meta, which owns Facebook and Instagram, said it planned to give outside researchers more detailed information on how political ads were targeted across its platform, providing insight into the ways that politicians, campaign operatives and political strategists buy and use ads ahead of the midterm elections.
  • The information includes which interest categories — such as “people who like dogs” or “people who enjoy the outdoors” — were chosen to aim an ad at someone.
  • While Meta has given outsiders some access into how its political ads were used in the past, it has restricted the amount of information that could be seen, citing privacy reasons.
  • ...3 more annotations...
  • “By making advertiser targeting criteria available for analysis and reporting on ads run about social issues, elections and politics, we hope to help people better understand the practices used to reach potential voters on our technologies,” the company said in a statement.
  • Meta said it had been bound by privacy rules and regulations on what types of data it could share with outsiders. In an interview, Jeff King, a vice president in Meta’s business integrity unit, said the company had hired thousands of workers over the past few years to review those privacy issues.
  • “Every single thing we release goes through a privacy review now,” he said. “We want to make sure we give people the right amount of data, but still remain privacy conscious while we do it.”
Javier E

How to Raise a University's Profile: Pricing and Packaging - NYTimes.com - 0 views

  • I talked to a half-dozen of Hugh Moren’s fellow students. A highly indebted senior who was terrified of the weak job market described George Washington, where he had invested considerable time getting and doing internships, as “the world’s most expensive trade school.” Another mentioned the abundance of rich students whose parents were giving them a fancy-sounding diploma the way they might a new car. There are serious students here, he acknowledged, but: “You can go to G.W. and essentially buy a degree.”
  • A recent study from the Organization for Economic Cooperation and Development found that, on average, American college graduates score well below college graduates from most other industrialized countries in mathematics. In literacy (“understanding, evaluating, using and engaging with written text”), scores are just average. This comes on the heels of Richard Arum and Josipa Roksa’s “Academically Adrift,” a study that found “limited or no learning” among many college students.Instead of focusing on undergraduate learning, nu
  • colleges have been engaged in the kind of building spree I saw at George Washington. Recreation centers with world-class workout facilities and lazy rivers rise out of construction pits even as students and parents are handed staggeringly large tuition bills. Colleges compete to hire famous professors even as undergraduates wander through academic programs that often lack rigor or coherence. Campuses vie to become the next Harvard — or at least the next George Washington — while ignoring the growing cost and suspect quality of undergraduate education.
  • ...58 more annotations...
  • John Silber embarked on a huge building campaign while bringing luminaries like Saul Bellow and Elie Wiesel on board to teach and lend their prestige to the B.U. name, creating a bigger, more famous and much more costly institution. He had helped write a game plan for the aspiring college president.
  • the American research university had evolved into a complicated and somewhat peculiar organization. It was built to be all things to all people: to teach undergraduates, produce knowledge, socialize young men and women, train workers for jobs, anchor local economies, even put on weekend sports events. And excellence was defined by similarity to old, elite institutions. Universities were judged by the quality of their scholars, the size of their endowments, the beauty of their buildings and the test scores of their incoming students.
  • Mr. Trachtenberg understood the centrality of the university as a physical place. New structures were a visceral sign of progress. They told visitors, donors and civic leaders that the institution was, like beams and scaffolding rising from the earth, ascending. He added new programs, recruited more students, and followed the dictate of constant expansion.
  • GWU is, for all intents and purposes, a for-profit organization. Best example: study abroad. Their top program, a partnering with Sciences Po, costs each student (30 of them, on a program with 'prestige' status?) a full semester's tuition. It costs GW, according to Sciences Po website, €1000. A neat $20,000 profit per student (who is in digging her/himself deeper and deeper in debt.) Moreover, the school takes a $500 admin fee for the study abroad application! With no guarantee that all credits transfer. Students often lose a partial semester, GW profits again. Nor does GW offer help with an antiquated, one-shot/no transfers, tricky registration process. It's tough luck in gay Paris.Just one of many examples. Dorms with extreme mold, off-campus housing impossible for freshmen and sophomores. Required meal plan: Chick-o-Filet etc. Classes with over 300 students (required).This is not Harvard, but costs same.Emotional problems? Counselors too few. Suicides continue and are not appropriately addressed. Caring environment? Extension so and so, please hold.It's an impressive campus, I'm an alum. If you apply, make sure the DC experience is worth the price: good are internships, a few colleges like Elliot School, post-grad.GWU uses undergrad $$ directly for building projects, like the medical center to which students have NO access. (Student health facility is underfunded, outsourced.)Outstanding professors still make a difference. But is that enough?
  • Mr. Trachtenberg, however, understood something crucial about the modern university. It had come to inhabit a market for luxury goods. People don’t buy Gucci bags merely for their beauty and functionality. They buy them because other people will know they can afford the price of purchase. The great virtue of a luxury good, from the manufacturer’s standpoint, isn’t just that people will pay extra money for the feeling associated with a name brand. It’s that the high price is, in and of itself, a crucial part of what people are buying.
  • Mr. Trachtenberg convinced people that George Washington was worth a lot more money by charging a lot more money. Unlike most college presidents, he was surprisingly candid about his strategy. College is like vodka, he liked to explain.
  • The Absolut Rolex plan worked. The number of applicants surged from some 6,000 to 20,000, the average SAT score of students rose by nearly 200 points, and the endowment jumped from $200 million to almost $1 billion.
  • The university became a magnet for the children of new money who didn’t quite have the SATs or family connections required for admission to Stanford or Yale. It also aggressively recruited international students, rich families from Asia and the Middle East who believed, as nearly everyone did, that American universities were the best in the world.
  • U.S. News & World Report now ranks the university at No. 54 nationwide, just outside the “first tier.”
  • The watch and vodka analogies are correct. Personally, I used car analogies when discussing college choices with my kids. We were in the fortunate position of being able to comfortably send our kids to any college in the country and have them leave debt free. Notwithstanding, I told them that they would be going to a state school unless they were able to get into one of about 40 schools that I felt, in whatever arbitrary manner I decided, that was worth the extra cost. They both ended up going to state schools.College is by and large a commodity and you get out of it what you put into it. Both of my kids worked hard in college and were involved in school life. They both left the schools better people and the schools better schools for them being there. They are both now successful adults.I believe too many people look for the prestige of a named school and that is not what college should be primarily about.
  • In 2013, only 14 percent of the university’s 10,000 undergraduates received a grant — a figure on a par with elite schools but far below the national average. The average undergraduate borrower leaves with about $30,800 in debt.
  • When I talk to the best high school students in my state I always stress the benefits of the honors college experience at an affordable public university. For students who won't qualify for a public honors college. the regular pubic university experience is far preferable to the huge debt of places like GW.
  • Carey would do well to look beyond high ticket private universities (which after all are still private enterprises) and what he describes as the Olympian heights of higher education (which for some reason seems also to embitter him) and look at the system overall . The withdrawal of public support was never a policy choice; it was a political choice, "packaged and branded" as some tax cutting palaver all wrapped up in the argument that a free-market should decide how much college should cost and how many seats we need. In such an environment, trustees at private universities are no more solely responsible for turning their degrees into commodities than the administrations of state universities are for raising the number of out-of-state students in order to offset the loss of support from their legislatures. No doubt, we will hear more about market based solutions and technology from Mr. Carey
  • I went to GW back in the 60s. It was affordable and it got me away from home in New York. While I was there, Newsweek famously published a article about the DC Universities - GW, Georgetown, American and Catholic - dubbing them the Pony league, the schools for the children of wealthy middle class New Yorkers who couldn't get into the Ivy League. Nobody really complained. But that wasn't me. I went because I wanted to be where the action was in the 60s, and as we used to say - "GW was literally a stone's throw from the White House. And we could prove it." Back then, the two biggest alumni names were Jackie Kennedy, who's taken some classes there, and J. Edgar Hoover. Now, according to the glossy magazine they send me each month, it's the actress Kerry Washington. There's some sort of progress there, but I'm a GW alum and not properly trained to understand it.
  • More and more, I notice what my debt-financed contributions to the revenue streams of my vendors earn them, not me. My banks earned enough to pay ridiculous bonuses to employees for reckless risk-taking. My satellite tv operator earned enough to overpay ESPN for sports programming that I never watch--and that, in turn, overpays these idiotic pro athletes and college sports administrators. My health insurer earned enough to defeat one-payor insurance; to enable the opaque, inefficient billing practices of hospitals and other providers; and to feed the behemoth pharmaceutical industry. My church earned enough to buy the silence of sex abuse victims and oppose progressive political candidates. And my govt earned enough to continue ag subsidies, inefficient defense spending, and obsolete transportation and energy policies.
  • I attended nearby Georgetown University and graduated in 1985. Relative to state schools and elite schools, it was expensive then. I took out loans. I had Pell grants. I had work-study and GSL. I paid my debt of $15,000 off in ten years. Would I have done it differently? Yes: I would have continued on to graduate school and not worried about paying off those big loans right after college. My career work out and I am grateful for the education I received and paid for. But I would not recommend to my nieces and nephews debts north of $100,000 for a BA in liberal arts. Go community. Then go state. Then punch your ticket to Harvard, Yale or Stanford — if you are good enough.
  • American universities appear to have more and more drifted away from educating individuals and citizens to becoming high priced trade schools and purveyors of occupational licenses. Lost in the process is the concept of expanding a student's ability to appreciate broadly and deeply, as well as the belief that a republican democracy needs an educated citizenry, not a trained citizenry, to function well.Both the Heisman Trophy winner and the producer of a successful tech I.P.O. likely have much in common, a college education whose rewards are limited to the financial. I don't know if I find this more sad on the individual level or more worrisome for the future of America.
  • This is now a consumer world for everything, including institutions once thought to float above the Shakespearean briars of the work-a-day world such as higher education, law and medicine. Students get this. Parents get this. Everything is negotiable: financial aid, a spot in the nicest dorm, tix to the big game. But through all this, there are faculty - lots of 'em - who work away from the fluff to link the ambitions of the students with the reality and rigor of the 21st century. The job of the student is to get beyond the visible hype of the surroundings and find those faculty members. They will make sure your investment is worth it
  • My experience in managing or working with GW alumni in their 20's or 30's has not been good. Virtually all have been mentally lazy and/or had a stunning sense of entitlement. Basically they've been all talk and no results. That's been quite a contrast to the graduates from VA/MD state universities.
  • This explains a lot of the modern, emerging mentality. It encompasses the culture of enforced grade inflation, cheating and anti-intellectualism in much of higher education. It is consistent with our culture of misleading statistics and information, cronyism and fake quality, the "best and the brightest" being only schemers and glad handers. The wisdom and creativity engendered by an honest, rigorous academic education are replaced by the disingenuous quick fix, the winner-take-all mentality that neglects the common good.
  • as the parent of GWU freshman I am grateful for every opportunity afforded her. She has a generous merit scholarship, is in the honors program with some small classes, and has access to internships that can be done while at school. GWU also gave her AP credits to advance her to sophomore status. Had she attended the state flagship school (where she was accepted into that exclusive honors program) she would have a great education but little else. It's not possible to do foreign affairs related internship far from D.C. or Manhattan. She went to a very competitive high school where for the one or two ivy league schools in which she was interested, she didn't have the same level of connections or wealth as many of her peers. Whether because of the Common Application or other factors, getting into a good school with financial help is difficult for a middle class student like my daughter who had a 4.0 GPA and 2300 on the SAT. She also worked after school.The bottom line - GWU offered more money than perceived "higher tier" universities, and brought tuition to almost that of our state school system. And by the way, I think she is also getting a very good education.
  • This article reinforces something I have learned during my daughter's college application process. Most students choose a school based on emotion (reputation) and not value. This luxury good analogy holds up.
  • The entire education problem can be solved by MOOCs lots and lots of them plus a few closely monitored tests and personal interviews with people. Of course many many people make MONEY off of our entirely inefficient way of "educating" -- are we even really doing that -- getting a degree does NOT mean one is actually educated
  • As a first-generation college graduate I entered GW ambitious but left saddled with debt, and crestfallen at the hard-hitting realization that my four undergraduate years were an aberration from what life is actually like post-college: not as simple as getting an [unpaid] internship with a fancy titled institution, as most Colonials do. I knew how to get in to college, but what do you do after the recess of life ends?I learned more about networking, resume plumping (designated responses to constituents...errr....replied to emails), and elevator pitches than actual theory, economic principles, strong writing skills, critical thinking, analysis, and philosophy. While relatively easy to get a job after graduating (for many with a GW degree this is sadly not the case) sustaining one and excelling in it is much harder. It's never enough just to be able to open a new door, you also need to be prepared to navigate your way through that next opportunity.
  • this is a very telling article. Aimless and directionless high school graduates are matched only by aimless and directionless institutes of higher learning. Each child and each parent should start with a goal - before handing over their hard earned tuition dollars, and/or leaving a trail of broken debt in the aftermath of a substandard, unfocused education.
  • it is no longer the most expensive university in America. It is the 46th.Others have been implementing the Absolut Rolex Plan. John Sexton turned New York University into a global higher-education player by selling the dream of downtown living to students raised on “Sex and the City.” Northeastern followed Boston University up the ladder. Under Steven B. Sample, the University of Southern California became a U.S. News top-25 university. Washington University in St. Louis did the same.
  • I currently attend GW, and I have to say, this article completely misrepresents the situation. I have yet to meet a single person who is paying the full $60k tuition - I myself am paying $30k, because the school gave me $30k in grants. As for the quality of education, Foreign Policy rated GW the #8 best school in the world for undergraduate education in international affairs, Princeton Review ranks it as one of the best schools for political science, and U.S. News ranks the law school #20. The author also ignores the role that an expanding research profile plays in growing a university's prestige and educational power.
  • it makes me very sad to see how expensive some public schools have become. Used to be you could work your way through a public school without loans, but not any more. Like you, I had the advantage of a largely-scholarship paid undergraduate education at a top private college. However, I was also offered a virtually free spot in my state university's (then new) honors college
  • this is the same playbook used by hospitals the past 30 years or so. It is how Hackensack Hospital became Hackensack Medical Center and McComb Hospital became Southwest Mississippi Regional Medical Center. No wonder the results have been the same in healthcare and higher education; both have priced themselves out of reach for average Americans.
  • a world where a college is rated not by the quality of its output, but instaed, by the quality of its inputs. A world where there is practically no work to be done by the administration because the college's reputation is made before the first class even begins! This is isanity! But this is the swill that the mammoth college marketing departments nationwide have shoved down America's throat. Colleges are ranked not by the quality of their graduates, but rather, by the test scores of their incoming students!
  • The Pew Foundation has been doing surveys on what students learn, how much homework they do, how much time they spend with professors etc. All good stuff to know before a student chooses a school. It is called the National Survey of Student Engagement (NSSE - called Nessy). It turns out that the higher ranked schools do NOT allow their information to be released to the public. It is SECRET.Why do you think that is?
  • The article blames "the standard university organizational model left teaching responsibilities to autonomous academic departments and individual faculty members, each of which taught and tested in its own way." This is the view of someone who has never taught at a university, nor thought much about how education there actually happens. Once undergraduates get beyond the general requirements, their educations _have_ to depend on "autonomous departments" because it's only those departments know what the requirements for given degree can be, and can grant the necessary accreditation of a given student. The idea that some administrator could know what's necessary for degrees in everything from engineering to fiction writing is nonsense, except that's what the people who only know the theory of education (but not its practice) actually seem to think. In the classroom itself, you have tremendously talented people, who nevertheless have their own particular strengths and approaches. Don't you think it's a good idea to let them do what they do best rather than trying to make everyone teach the same way? Don't you think supervision of young teachers by older colleagues, who actually know their field and its pedagogy, rather than some administrator, who knows nothing of the subject, is a good idea?
  • And in hundreds of regional universities and community colleges, presidents and deans and department chairmen have watched this spectacle of ascension and said to themselves, “That could be me.” Agricultural schools and technical institutes are lobbying state legislatures for tuition increases and Ph.D. programs, fitness centers and arenas for sport. Presidents and boards are drawing up plans to raise tuition, recruit “better” students and add academic programs. They all want to go in one direction — up! — and they are all moving with a single vision of what they want to be.
  • My daughter attended a good community college for a couple of classes during her senior year of high school and I could immediately see how such places are laboratories for failure. They seem like high schools in atmosphere and appearance. Students rush in by car and rush out again when the class is over.The four year residency college creates a completely different feel. On arrival, you get the sense that you are engaging in something important, something apart and one that will require your full attention. I don't say this is for everyone or that the model is not flawed in some ways (students actually only spend 2 1/2 yrs. on campus to get the four yr. degree). College is supposed to be a 60 hour per week job. Anything less than that and the student is seeking himself or herself
  • This. Is. STUNNING. I have always wondered, especially as my kids have approached college age, why American colleges have felt justified in raising tuition at a rate that has well exceeded inflation, year after year after year. (Nobody needs a dorm with luxury suites and a lazy river pool at college!) And as it turns out, they did it to become luxury brands. Just that simple. Incredible.I don't even blame this guy at GWU for doing what he did. He wasn't made responsible for all of American higher ed. But I do think we all need to realize what happened, and why. This is front page stuff.
  • I agree with you, but, unfortunately, given the choice between low tuition, primitive dorms, and no athletic center VS expensive & luxurious, the customers (and their parents) are choosing the latter. As long as this is the case, there is little incentive to provide bare-bones and cheap education.
  • Wesleyan University in CT is one school that is moving down the rankings. Syracuse University is another. Reed College is a third. Why? Because these schools try hard to stay out of the marketing game. (With its new president, Syracuse has jumped back into the game.) Bryn Mawr College, outside Philadelphia hasn't fared well over the past few decades in the rankings, which is true of practically every women's college. Wellesley is by far the highest ranked women's college, but even there the acceptance rate is significantly higher than one finds at comparable coed liberal arts colleges like Amherst & Williams. University of Chicago is another fascinating case for Mr. Carey to study (I'm sure he does in his forthcoming book, which I look forward to reading). Although it has always enjoyed an illustrious academic reputation, until recently Chicago's undergraduate reputation paled in comparison to peer institutions on the two coasts. A few years ago, Chicago changed its game plan to more closely resemble Harvard and Stanford in undergraduate amenities, and lo and behold, its rankings shot up. It was a very cynical move on the president's part to reassemble the football team, but it was a shrewd move because athletics draw more money than academics ever can (except at engineering schools like Cal Tech & MIT), and more money draws richer students from fancier secondary schools with higher test scores, which lead to higher rankings - and the beat goes on.
  • College INDUSTRY is out of control. Sorry, NYU, GW, BU are not worth the price. Are state schools any better? We have the University of Michigan, which is really not a state school, but a university that gives a discount to people who live in Michigan. Why? When you have an undergraduate body 40+% out-of-state that pays tuition of over $50K/year, you tell me?Perhaps the solution is two years of community college followed by two at places like U of M or Michigan State - get the same diploma at the end for much less and beat the system.
  • In one recent yr., the majority of undergrad professors at Harvard, according to Boston.com, where adjuncts. That means low pay, no benefits, no office, temp workers. Harvard.Easily available student loans fueled this arms race of amenities and frills that in which colleges now engage. They moved the cost of education onto the backs of people, kids, who don't understand what they are doing.Students in colleges these days are customers and the customers must be able to get through. If it requires dumbing things down, so be it. On top of tuition, G.W. U. is known by its students as the land of added fees on top of added fees. The joke around campus was that they would soon be installing pay toilets in the student union. No one was laughing.
  • You could written the same story about my alma mater, American University. The place reeked of ambition and upward mobility decades ago and still does. Whoever's running it now must look at its measly half-billion-dollar endowment and compare it to GWU's $1.5 billion and seethe with envy, while GWU's president sets his sights on an Ivy League-size endowment. And both get back to their real jobs: 24/7 fundraising,Which is what university presidents are all about these days. Money - including million-dollar salaries for themselves (GWU's president made more than Harvard's in 2011) - pride, cachet, power, a mansion, first-class all the way. They should just be honest about it and change their university's motto to Ostende mihi pecuniam! (please excuse my questionable Latin)Whether the students are actually learning anything is up to them, I guess - if they do, it's thanks to the professors, adjuncts and the administrative staff, who do the actual work of educating and keep the school running.
  • When I was in HS (70s), many of my richer friends went to GW and I was then of the impression that GW was a 'good' school. As I age, I have come to realize that this place is just another façade to the emptiness that has become America. All too often are we faced with a dilemma: damned if we do, damned if we don't. Yep, 'education' has become a trap for all too many of our citizen.
  • I transferred to GWU from a state school. I am forever grateful that I did. I wanted to get a good rigorous education and go to one of the best International Affairs schools in the world. Even though the state school I went to was dirt-cheap, the education and the faculty was awful. I transferred to GW and was amazed at the professors at that university. An ambassador or a prominent IA scholar taught every class. GW is an expensive school, but that is the free market. If you want a good education you need to be willing to pay for it or join the military. I did the latter and my school was completely free with no debt and I received an amazing education. If young people aren't willing to make some sort of sacrifice to get ahead or just expect everything to be given to then our country is in a sad state.We need to stop blaming universities like GWU that strive to attract better students, better professors, and better infrastructure. They are doing what is expected in America, to better oneself.
  • "Whether the students are actually learning anything is up to them, I guess." How could it possibly be otherwise??? I am glad that you are willing to give credit to teachers and administrators, but it is not they who "do the actual work of educating." From this fallacy comes its corollary, that we should blame teachers first for "under-performing schools". This long-running show of scapegoating may suit the wallets and vanity of American parents, but it is utterly senseless. When, if ever, American culture stops reeking of arrogance, greed and anti-intellectualism, things may improve, and we may resume the habit of bothering to learn. Until then, nothing doing.
  • Universities sell knowledge and grade students on how much they have learned. Fundamentally, there is conflict of interest in thsi setup. Moreover, students who are poorly educated, even if they know this, will not criticize their school, because doing so would make it harder for them to have a career. As such, many problems with higher education remain unexposed to the public.
  • I've lectured and taught in at least five different countries in three continents and the shortest perusal of what goes on abroad would totally undermine most of these speculations. For one thing American universities are unique in their dedication to a broad based liberal arts type education. In France, Italy or Germany, for example, you select a major like mathematics or physics and then in your four years you will not take even one course in another subject. The amount of work that you do that is critically evaluated by an instructor is a tiny fraction of what is done in an American University. While half educated critics based on profoundly incomplete research write criticism like this Universities in Germany Italy, the Netherlands, South Korea and Japan as well as France have appointed committees and made studies to explain why the American system of higher education so drastically outperforms their own system. Elsewhere students do get a rather nice dose of general education but it ends in secondary school and it has the narrowness and formulaic quality that we would just normally associate with that. The character who wrote this article probably never set foot on a "campus" of the University of Paris or Rome
  • This is the inevitable result of privatizing higher education. In the not-so-distant past, we paid for great state universities through our taxes, not tuition. Then, the states shifted funding to prisons and the Federal government radically cut research support and the GI bill. Instead, today we expect universities to support themselves through tuition, and to the extent that we offered students support, it is through non-dischargeable loans. To make matters worse, the interest rates on those loans are far above the government's cost of funds -- so in effect the loans are an excise tax on education (most of which is used to support a handful of for-profit institutions that account for the most student defaults). This "consumer sovereignty" privatized model of funding education works no better than privatizing California's electrical system did in the era of Enron, or our privatized funding of medical service, or our increasingly privatized prison system: it drives up costs at the same time that it replace quality with marketing.
  • The university is part of a complex economic system and it is responding to the demands of that system. For example, students and parents choose universities that have beautiful campuses and buildings. So universities build beautiful campuses. State support of universities has greatly declined, and this decline in funding is the greatest cause of increased tuition. Therefore universities must compete for dollars and must build to attract students and parents. Also, universities are not ranked based on how they educate students -- that's difficult to measure so it is not measured. Instead universities are ranked on research publications. So while universities certainly put much effort into teaching, research has to have a priority in order for the university to survive. Also universities do not force students and parents to attend high price institutions. Reasonably priced state institutions and community colleges are available to every student. Community colleges have an advantage because they are funded by property taxes. Finally learning requires good teaching, but it also requires students that come to the university funded, prepared, and engaged. This often does not happen. Conclusion- universities have to participate in profile raising actions in order to survive. The day that funding is provided for college, ranking is based on education, and students choose campuses with simple buildings, then things will change at the university.
  • There are data in some instances on student learning, but the deeper problem, as I suspect the author already knows, is that there is nothing like a consensus on how to measure that learning, or even on when is the proper end point to emphasize (a lot of what I teach -- I know this from what students have told me -- tends to come into sharp focus years after graduation).
  • Michael (Baltimore) has hit the nail on the head. Universities are increasingly corporatized institutions in the credentialing business. Knowledge, for those few who care about it (often not those paying for the credentials) is available freely because there's no profit in it. Like many corporate entities, it is increasingly run by increasingly highly paid administrators, not faculty.
  • GWU has not defined itself in any unique way, it has merely embraced the bland, but very expensive, accoutrements of American private education: luxury dorms, food courts, spa-like gyms, endless extracurricular activities, etc. But the real culprit for this bloat that students have to bear financially is the college ranking system by US News, Princeton Review, etc. An ultimately meaningless exercise in competition that has nevertheless pushed colleges and universities to be more like one another. A sad state of affairs, and an extremely expensive one for students
  • It is long past time to realize the failure of the Reagonomics-neoliberal private profits over public good program. In education, we need to return to public institutions publicly funded. Just as we need to recognize that Medicare, Social Security, the post office, public utilities, fire departments, interstate highway system, Veterans Administration hospitals and the GI bill are models to be improved and expanded, not destroyed.
  • George Washington is actually not a Rolex watch, it is a counterfeit Rolex. The real Rolexes of higher education -- places like Hopkins, Georgetown, Duke, the Ivies etc. -- have real endowments and real financial aid. No middle class kid is required to borrow $100,000 to get a degree from those schools, because they offer generous need-based financial aid in the form of grants, not loans. The tuition at the real Rolexes is really a sticker price that only the wealthy pay -- everybody else on a sliding scale. For middle class kids who are fortunate enough to get in, Penn actually ends up costing considerably less than a state university.The fake Rolexes -- BU, NYU, Drexel in Philadelphia -- don't have the sliding scale. They bury middle class students in debt.And really, though it is foolish to borrow $100,000 or $120,000 for an undergraduate degree, I don't find the transaction morally wrong. What is morally wrong is our federal government making that loan non-dischargeable in bankruptcy, so many if these kids will be having their wages garnished for the REST OF THEIR LIVES.There is a very simple solution to this, by the way. Cap the amount of non-dischargeable student loan debt at, say, $50,000
  • The slant of this article is critical of the growth of research universities. Couldn't disagree more. Modern research universities create are incredibly engines of economic opportunity not only for the students (who pay the bills) but also for the community via the creation of blue and white collar jobs. Large research university employ tens of thousands of locals from custodial and food service workers right up to high level administrators and specialist in finance, computer services, buildings and facilities management, etc. Johns Hopkins University and the University of Maryland system employ more people than any other industry in Maryland -- including the government. Research universities typically have hospitals providing cutting-edge medical care to the community. Local business (from cafes to property rental companies) benefit from a built-in, long-term client base as well as an educated workforce. And of course they are the foundry of new knowledge which is critical for the future growth of our country.Check out the work of famed economist Dr. Julia Lane on modeling the economic value of the research university. In a nutshell, there are few better investments America can make in herself than research universities. We are the envy of the world in that regard -- and with good reason. How many *industries* (let alone jobs) have Stanford University alone catalyzed?
  • What universities have the monopoly on is the credential. Anyone can learn, from books, from free lectures on the internet, from this newspaper, etc. But only universities can endow you with the cherished degree. For some reason, people are will to pay more for one of these pieces of paper with a certain name on it -- Ivy League, Stanford, even GW -- than another -- Generic State U -- though there is no evidence one is actually worth more in the marketplace of reality than the other. But, by the laws of economics, these places are actually underpriced: after all, something like 20 times more people are trying to buy a Harvard education than are allowed to purchase one. Usually that means you raise your price.
  • Overalll a good article, except for - "This comes on the heels of Richard Arum and Josipa Roksa’s “Academically Adrift,” a study that found “limited or no learning” among many college students." The measure of learning you report was a general thinking skills exam. That's not a good measure of college gains. Most psychologists and cognitive scientists worth their salt would tell you that improvement in critical thinking skills is going to be limited to specific areas. In other words, learning critical thinking skills in math will make little change in critical thinking about political science or biology. Thus we should not expect huge improvements in general critical thinking skills, but rather improvements in a student's major and other areas of focus, such as a minor. Although who has time for a minor when it is universally acknowledged that the purpose of a university is to please and profit an employer or, if one is lucky, an investor. Finally, improved critical thinking skills are not the end all and be all of a college education even given this profit centered perspective. Learning and mastering the cumulative knowledge of past generations is arguably the most important thing to be gained, and most universities still tend to excel at that even with the increasing mandate to run education like a business and cultivate and cull the college "consumer".
  • As for community colleges, there was an article in the Times several years ago that said it much better than I could have said it myself: community colleges are places where dreams are put on hold. Without making the full commitment to study, without leaving the home environment, many, if not most, community college students are caught betwixt and between, trying to balance work responsibilities, caring for a young child or baby and attending classes. For males, the classic "end of the road" in community college is to get a car, a job and a girlfriend, one who is not in college, and that is the end of the dream. Some can make it, but most cannot.
  • as a scientist I disagree with the claim that undergrad tuition subsidizes basic research. Nearly all lab equipment and research personnel (grad students, technicians, anyone with the title "research scientist" or similar) on campus is paid for through federal grants. Professors often spend all their time outside teaching and administration writing grant proposals, as the limited federal grant funds mean ~%85 of proposals must be rejected. What is more, out of each successful grant the university levies a "tax", called "overhead", of 30-40%, nominally to pay for basic operations (utilities, office space, administrators). So in fact one might say research helps fund the university rather than the other way around. Flag
  • It's certainly overrated as a research and graduate level university. Whether it is good for getting an undergraduate education is unclear, but a big part of the appeal is getting to live in D.C..while attending college instead of living in some small college town in the corn fields.
Javier E

The decline effect and the scientific method : The New Yorker - 3 views

  • The test of replicability, as it’s known, is the foundation of modern research. Replicability is how the community enforces itself. It’s a safeguard for the creep of subjectivity. Most of the time, scientists know what results they want, and that can influence the results they get. The premise of replicability is that the scientific community can correct for these flaws.
  • But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable.
  • This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology.
  • ...39 more annotations...
  • If replication is what separates the rigor of science from the squishiness of pseudoscience, where do we put all these rigorously validated findings that can no longer be proved? Which results should we believe?
  • Schooler demonstrated that subjects shown a face and asked to describe it were much less likely to recognize the face when shown it later than those who had simply looked at it. Schooler called the phenomenon “verbal overshadowing.”
  • The most likely explanation for the decline is an obvious one: regression to the mean. As the experiment is repeated, that is, an early statistical fluke gets cancelled out. The extrasensory powers of Schooler’s subjects didn’t decline—they were simply an illusion that vanished over time.
  • yet Schooler has noticed that many of the data sets that end up declining seem statistically solid—that is, they contain enough data that any regression to the mean shouldn’t be dramatic. “These are the results that pass all the tests,” he says. “The odds of them being random are typically quite remote, like one in a million. This means that the decline effect should almost never happen. But it happens all the time!
  • this is why Schooler believes that the decline effect deserves more attention: its ubiquity seems to violate the laws of statistics
  • In 2001, Michael Jennions, a biologist at the Australian National University, set out to analyze “temporal trends” across a wide range of subjects in ecology and evolutionary biology. He looked at hundreds of papers and forty-four meta-analyses (that is, statistical syntheses of related studies), and discovered a consistent decline effect over time, as many of the theories seemed to fade into irrelevance.
  • Jennions admits that his findings are troubling, but expresses a reluctance to talk about them
  • publicly. “This is a very sensitive issue for scientists,” he says. “You know, we’re supposed to be dealing with hard facts, the stuff that’s supposed to stand the test of time. But when you see these trends you become a little more skeptical of things.”
  • Sterling saw that if ninety-seven per cent of psychology studies were proving their hypotheses, either psychologists were extraordinarily lucky or they published only the outcomes of successful experiments.
  • Jennions, similarly, argues that the decline effect is largely a product of publication bias, or the tendency of scientists and scientific journals to prefer positive data over null results, which is what happens when no effect is found. The bias was first identified by the statistician Theodore Sterling, in 1959, after he noticed that ninety-seven per cent of all published psychological studies with statistically significant data found the effect they were looking for
  • While publication bias almost certainly plays a role in the decline effect, it remains an incomplete explanation. For one thing, it fails to account for the initial prevalence of positive results among studies that never even get submitted to journals. It also fails to explain the experience of people like Schooler, who have been unable to replicate their initial data despite their best efforts.
  • One of his most cited papers has a deliberately provocative title: “Why Most Published Research Findings Are False.”
  • suspects that an equally significant issue is the selective reporting of results—the data that scientists choose to document in the first place. Palmer’s most convincing evidence relies on a statistical tool known as a funnel graph. When a large number of studies have been done on a single subject, the data should follow a pattern: studies with a large sample size should all cluster around a common value—the true result—whereas those with a smaller sample size should exhibit a random scattering, since they’re subject to greater sampling error. This pattern gives the graph its name, since the distribution resembles a funnel.
  • after Palmer plotted every study of fluctuating asymmetry, he noticed that the distribution of results with smaller sample sizes wasn’t random at all but instead skewed heavily toward positive results. Palmer has since documented a similar problem in several other contested subject areas. “Once I realized that selective reporting is everywhere in science, I got quite depressed,” Palmer told me. “As a researcher, you’re always aware that there might be some nonrandom patterns, but I had no idea how widespread it is.”
  • Palmer summarized the impact of selective reporting on his field: “We cannot escape the troubling conclusion that some—perhaps many—cherished generalities are at best exaggerated in their biological significance and at worst a collective illusion nurtured by strong a-priori beliefs often repeated.”
  • Palmer emphasizes that selective reporting is not the same as scientific fraud. Rather, the problem seems to be one of subtle omissions and unconscious misperceptions, as researchers struggle to make sense of their results. Stephen Jay Gould referred to this as the “sho
  • horning” process.
  • “A lot of scientific measurement is really hard,” Simmons told me. “If you’re talking about fluctuating asymmetry, then it’s a matter of minuscule differences between the right and left sides of an animal. It’s millimetres of a tail feather. And so maybe a researcher knows that he’s measuring a good male”—an animal that has successfully mated—“and he knows that it’s supposed to be symmetrical. Well, that act of measurement is going to be vulnerable to all sorts of perception biases. That’s not a cynical statement. That’s just the way human beings work.”
  • One of the classic examples of selective reporting concerns the testing of acupuncture in different countries. While acupuncture is widely accepted as a medical treatment in various Asian countries, its use is much more contested in the West. These cultural differences have profoundly influenced the results of clinical trials.
  • John Ioannidis, an epidemiologist at Stanford University, argues that such distortions are a serious issue in biomedical research. “These exaggerations are why the decline has become so common,” he says. “It’d be really great if the initial studies gave us an accurate summary of things. But they don’t. And so what happens is we waste a lot of money treating millions of patients and doing lots of follow-up studies on other themes based on results that are misleading.”
  • In 2005, Ioannidis published an article in the Journal of the American Medical Association that looked at the forty-nine most cited clinical-research studies in three major medical journals.
  • the data Ioannidis found were disturbing: of the thirty-four claims that had been subject to replication, forty-one per cent had either been directly contradicted or had their effect sizes significantly downgraded.
  • the most troubling fact emerged when he looked at the test of replication: out of four hundred and thirty-two claims, only a single one was consistently replicable. “This doesn’t mean that none of these claims will turn out to be true,” he says. “But, given that most of them were done badly, I wouldn’t hold my breath.”
  • According to Ioannidis, the main problem is that too many researchers engage in what he calls “significance chasing,” or finding ways to interpret the data so that it passes the statistical test of significance—the ninety-five-per-cent boundary invented by Ronald Fisher.
  • For Simmons, the steep rise and slow fall of fluctuating asymmetry is a clear example of a scientific paradigm, one of those intellectual fads that both guide and constrain research: after a new paradigm is proposed, the peer-review process is tilted toward positive results. But then, after a few years, the academic incentives shift—the paradigm has become entrenched—so that the most notable results are now those that disprove the theory.
  • The problem of selective reporting is rooted in a fundamental cognitive flaw, which is that we like proving ourselves right and hate being wrong.
  • “It feels good to validate a hypothesis,” Ioannidis said. “It feels even better when you’ve got a financial interest in the idea or your career depends upon it. And that’s why, even after a claim has been systematically disproven”—he cites, for instance, the early work on hormone replacement therapy, or claims involving various vitamins—“you still see some stubborn researchers citing the first few studies
  • That’s why Schooler argues that scientists need to become more rigorous about data collection before they publish. “We’re wasting too much time chasing after bad studies and underpowered experiments,”
  • The current “obsession” with replicability distracts from the real problem, which is faulty design.
  • “Every researcher should have to spell out, in advance, how many subjects they’re going to use, and what exactly they’re testing, and what constitutes a sufficient level of proof. We have the tools to be much more transparent about our experiments.”
  • Schooler recommends the establishment of an open-source database, in which researchers are required to outline their planned investigations and document all their results. “I think this would provide a huge increase in access to scientific work and give us a much better way to judge the quality of an experiment,”
  • scientific research will always be shadowed by a force that can’t be curbed, only contained: sheer randomness. Although little research has been done on the experimental dangers of chance and happenstance, the research that exists isn’t encouraging.
  • The disturbing implication of the Crabbe study is that a lot of extraordinary scientific data are nothing but noise. The hyperactivity of those coked-up Edmonton mice wasn’t an interesting new fact—it was a meaningless outlier, a by-product of invisible variables we don’t understand.
  • The problem, of course, is that such dramatic findings are also the most likely to get published in prestigious journals, since the data are both statistically significant and entirely unexpected
  • This suggests that the decline effect is actually a decline of illusion. While Karl Popper imagined falsification occurring with a single, definitive experiment—Galileo refuted Aristotelian mechanics in an afternoon—the process turns out to be much messier than that.
  • Many scientific theories continue to be considered true even after failing numerous experimental tests.
  • Even the law of gravity hasn’t always been perfect at predicting real-world phenomena. (In one test, physicists measuring gravity by means of deep boreholes in the Nevada desert found a two-and-a-half-per-cent discrepancy between the theoretical predictions and the actual data.)
  • Such anomalies demonstrate the slipperiness of empiricism. Although many scientific ideas generate conflicting results and suffer from falling effect sizes, they continue to get cited in the textbooks and drive standard medical practice. Why? Because these ideas seem true. Because they make sense. Because we can’t bear to let them go. And this is why the decline effect is so troubling. Not because it reveals the human fallibility of science, in which data are tweaked and beliefs shape perceptions. (Such shortcomings aren’t surprising, at least for scientists.) And not because it reveals that many of our most exciting theories are fleeting fads and will soon be rejected. (That idea has been around since Thomas Kuhn.)
  • The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true. When the experiments are done, we still have to choose what to believe. ♦
Javier E

Clouds' Effect on Climate Change Is Last Bastion for Dissenters - NYTimes.com - 0 views

  • For decades, a small group of scientific dissenters has been trying to shoot holes in the prevailing science of climate change, offering one reason after another why the outlook simply must be wrong. Enlarge This Image Josh Haner/The New York Times A technician at a Department of Energy site in Oklahoma launching a weather balloon to help scientists analyze clouds. More Photos » Temperature Rising Enigma in the Sky This series focuses on the central arguments in the climate debate and examining the evidence for global warming and its consequences. More From the Series » if (typeof NYTDVideoManager != "undefined") { NYTDVideoManager.setAllowMultiPlayback(false); } function displayCompanionBanners(banners, tracking) { tmDisplayBanner(banners, "videoAdContent", 300, 250, null, tracking); } Multimedia Interactive Graphic Clouds and Climate Slide Show Understanding the Atmosphere Related Green Blog: Climate Change and the Body Politic (May 1, 2012) An Underground Fossil Forest Offers Clues on Climate Change (May 1, 2012) A blog about energy and the environment. Go to Blog » Readers’ Comments "There is always some possibility that the scientific consensus may be wrong and Dr. Lindzen may be right, or that both may be wrong. But the worst possible place to resolve such issues is the political arena." Alexander Flax, Potomac, MD Read Full Comment » Post a Comment » Over time, nearly every one of their arguments has been knocked down by accumulating evidence, and polls say 97 percent of working climate scientists now see global warming as a serious risk.
  • They acknowledge that the human release of greenhouse gases will cause the planet to warm. But they assert that clouds — which can either warm or cool the earth, depending on the type and location — will shift in such a way as to counter much of the expected temperature rise and preserve the equable climate on which civilization depends.
  • At gatherings of climate change skeptics on both sides of the Atlantic, Dr. Lindzen has been treated as a star. During a debate in Australia over carbon taxes, his work was cited repeatedly. When he appears at conferences of the Heartland Institute, the primary American organization pushing climate change skepticism, he is greeted by thunderous applause.
  • ...13 more annotations...
  • His idea has drawn withering criticism from other scientists, who cite errors in his papers and say proof is lacking. Enough evidence is already in hand, they say, to rule out the powerful cooling effect from clouds that would be needed to offset the increase of greenhouse gases.
  • “If you listen to the credible climate skeptics, they’ve really pushed all their chips onto clouds.”
  • Dr. Lindzen is “feeding upon an audience that wants to hear a certain message, and wants to hear it put forth by people with enough scientific reputation that it can be sustained for a while, even if it’s wrong science,” said Christopher S. Bretherton, an atmospheric researcher at the University of Washington. “I don’t think it’s intellectually honest at all.”
  • With climate policy nearly paralyzed in the United States, many other governments have also declined to take action, and worldwide emissions of greenhouse gases are soaring.
  • The most elaborate computer programs have agreed on a broad conclusion: clouds are not likely to change enough to offset the bulk of the human-caused warming. Some of the analyses predict that clouds could actually amplify the warming trend sharply through several mechanisms, including a reduction of some of the low clouds that reflect a lot of sunlight back to space. Other computer analyses foresee a largely neutral effect. The result is a big spread in forecasts of future temperature, one that scientists have not been able to narrow much in 30 years of effort.
  • The earth’s surface has already warmed about 1.4 degrees Fahrenheit since the Industrial Revolution, most of that in the last 40 years. Modest as it sounds, it is an average for the whole planet, representing an enormous addition of heat. An even larger amount is being absorbed by the oceans. The increase has caused some of the world’s land ice to melt and the oceans to rise.
  • Even in the low projection, many scientists say, the damage could be substantial. In the high projection, some polar regions could heat up by 20 or 25 degrees Fahrenheit — more than enough, over centuries or longer, to melt the Greenland ice sheet, raising sea level by a catastrophic 20 feet or more. Vast changes in  rainfall, heat waves and other weather patterns would most likely accompany such a large warming. “The big damages come if the climate sensitivity to greenhouse gases turns out to be high,” said Raymond T. Pierrehumbert, a climate scientist at the University of Chicago. “Then it’s not a bullet headed at us, but a thermonuclear warhead.”
  • But the problem of how clouds will behave in a future climate is not yet solved — making the unheralded field of cloud research one of the most important pursuits of modern science.
  • for more than a decade, Dr. Lindzen has said that when surface temperature increases, the columns of moist air rising in the tropics will rain out more of their moisture, leaving less available to be thrown off as ice, which forms the thin, high clouds known as cirrus. Just like greenhouse gases, these cirrus clouds act to reduce the cooling of the earth, and a decrease of them would counteract the increase of greenhouse gases. Dr. Lindzen calls his mechanism the iris effect, after the iris of the eye, which opens at night to let in more light. In this case, the earth’s “iris” of high clouds would be opening to let more heat escape.
  • Dr. Lindzen acknowledged that the 2009 paper contained “some stupid mistakes” in his handling of the satellite data. “It was just embarrassing,” he said in an interview. “The technical details of satellite measurements are really sort of grotesque.” Last year, he tried offering more evidence for his case, but after reviewers for a prestigious American journal criticized the paper, Dr. Lindzen published it in a little-known Korean journal. Dr. Lindzen blames groupthink among climate scientists for his publication difficulties, saying the majority is determined to suppress any dissenting views. They, in turn, contend that he routinely misrepresents the work of other researchers.
  • Ultimately, as the climate continues warming and more data accumulate, it will become obvious how clouds are reacting. But that could take decades, scientists say, and if the answer turns out to be that catastrophe looms, it would most likely be too late. By then, they say, the atmosphere would contain so much carbon dioxide as to make a substantial warming inevitable, and the gas would not return to a normal level for thousands of years.
  • In his Congressional appearances, speeches and popular writings, Dr. Lindzen offers little hint of how thin the published science supporting his position is. Instead, starting from his disputed iris mechanism, he makes what many of his colleagues see as an unwarranted leap of logic, professing near-certainty that climate change is not a problem society needs to worry about.
  • “Even if there were no political implications, it just seems deeply unprofessional and irresponsible to look at this and say, ‘We’re sure it’s not a problem,’ ” said Kerry A. Emanuel, another M.I.T. scientist. “It’s a special kind of risk, because it’s a risk to the collective civilization.”
Javier E

Lies, Damned Lies, and Medical Science - Magazine - The Atlantic - 0 views

  • How should we choose among these dueling, high-profile nutritional findings? Ioannidis suggests a simple approach: ignore them all.
  • even if a study managed to highlight a genuine health connection to some nutrient, you’re unlikely to benefit much from taking more of it, because we consume thousands of nutrients that act together as a sort of network, and changing intake of just one of them is bound to cause ripples throughout the network that are far too complex for these studies to detect, and that may be as likely to harm you as help you
  • studies report average results that typically represent a vast range of individual outcomes.
  • ...17 more annotations...
  • studies usually detect only modest effects that merely tend to whittle your chances of succumbing to a particular disease from small to somewhat smaller
  • The odds that anything useful will survive from any of these studies are poor,” says Ioannidis—dismissing in a breath a good chunk of the research into which we sink about $100 billion a year in the United States alone.
  • nutritional studies aren’t the worst. Drug studies have the added corruptive force of financial conflict of interest.
  • Even when the evidence shows that a particular research idea is wrong, if you have thousands of scientists who have invested their careers in it, they’ll continue to publish papers on it,” he says. “It’s like an epidemic, in the sense that they’re infected with these wrong ideas, and they’re spreading it to other researchers through journals.
  • Nature, the grande dame of science journals, stated in a 2006 editorial, “Scientists understand that peer review per se provides only a minimal assurance of quality, and that the public conception of peer review as a stamp of authentication is far from the truth.
  • The ultimate protection against research error and bias is supposed to come from the way scientists constantly retest each other’s results—except they don’t. Only the most prominent findings are likely to be put to the test, because there’s likely to be publication payoff in firming up the proof, or contradicting it.
  • even for medicine’s most influential studies, the evidence sometimes remains surprisingly narrow. Of those 45 super-cited studies that Ioannidis focused on, 11 had never been retested
  • even when a research error is outed, it typically persists for years or even decades.
  • much, perhaps even most, of what doctors do has never been formally put to the test in credible studies, given that the need to do so became obvious to the field only in the 1990s
  • Other meta-research experts have confirmed that similar issues distort research in all fields of science, from physics to economics (where the highly regarded economists J. Bradford DeLong and Kevin Lang once showed how a remarkably consistent paucity of strong evidence in published economics studies made it unlikely that any of them were right
  • His PLoS Medicine paper is the most downloaded in the journal’s history, and it’s not even Ioannidis’s most-cited work
  • while his fellow researchers seem to be getting the message, he hasn’t necessarily forced anyone to do a better job. He fears he won’t in the end have done much to improve anyone’s health. “There may not be fierce objections to what I’m saying,” he explains. “But it’s difficult to change the way that everyday doctors, patients, and healthy people think and behave.”
  • “Usually what happens is that the doctor will ask for a suite of biochemical tests—liver fat, pancreas function, and so on,” she tells me. “The tests could turn up something, but they’re probably irrelevant. Just having a good talk with the patient and getting a close history is much more likely to tell me what’s wrong.” Of course, the doctors have all been trained to order these tests, she notes, and doing so is a lot quicker than a long bedside chat. They’re also trained to ply the patient with whatever drugs might help whack any errant test numbers back into line.
  • What they’re not trained to do is to go back and look at the research papers that helped make these drugs the standard of care. “When you look the papers up, you often find the drugs didn’t even work better than a placebo. And no one tested how they worked in combination with the other drugs,” she says. “Just taking the patient off everything can improve their health right away.” But not only is checking out the research another time-consuming task, patients often don’t even like it when they’re taken off their drugs, she explains; they find their prescriptions reassuring.
  • Already feeling that they’re fighting to keep patients from turning to alternative medical treatments such as homeopathy, or misdiagnosing themselves on the Internet, or simply neglecting medical treatment altogether, many researchers and physicians aren’t eager to provide even more reason to be skeptical of what doctors do—not to mention how public disenchantment with medicine could affect research funding.
  • We could solve much of the wrongness problem, Ioannidis says, if the world simply stopped expecting scientists to be right. That’s because being wrong in science is fine, and even necessary—as long as scientists recognize that they blew it, report their mistake openly instead of disguising it as a success, and then move on to the next thing, until they come up with the very occasional genuine breakthrough
  • Science is a noble endeavor, but it’s also a low-yield endeavor,” he says. “I’m not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life. We should be very comfortable with that fact.”
Javier E

Noted Dutch Psychologist, Stapel, Accused of Research Fraud - NYTimes.com - 0 views

  • A well-known psychologist in the Netherlands whose work has been published widely in professional journals falsified data and made up entire experiments, an investigating committee has found
  • Experts say the case exposes deep flaws in the way science is done in a field, psychology, that has only recently earned a fragile respectability.
  • In recent years, psychologists have reported a raft of findings on race biases, brain imaging and even extrasensory perception that have not stood up to scrutiny. Outright fraud may be rare, these experts say, but they contend that Dr. Stapel took advantage of a system that allows researchers to operate in near secrecy and massage data to find what they want to find, without much fear of being challenged.
  • ...8 more annotations...
  • “The big problem is that the culture is such that researchers spin their work in a way that tells a prettier story than what they really found,” said Jonathan Schooler, a psychologist at the University of California, Santa Barbara. “It’s almost like everyone is on steroids, and to compete you have to take steroids as well.”
  • Dr. Stapel published papers on the effect of power on hypocrisy, on racial stereotyping and on how advertisements affect how people view themselves. Many of his findings appeared in newspapers around the world, including The New York Times, which reported in December on his study about advertising and identity.
  • Dr. Stapel was able to operate for so long, the committee said, in large measure because he was “lord of the data,” the only person who saw the experimental evidence that had been gathered (or fabricated). This is a widespread problem in psychology, said Jelte M. Wicherts, a psychologist at the University of Amsterdam. In a recent survey, two-thirds of Dutch research psychologists said they did not make their raw data available for other researchers to see. “This is in violation of ethical rules established in the field,” Dr. Wicherts said.
  • In a survey of more than 2,000 American psychologists scheduled to be published this year, Leslie John of Harvard Business School and two colleagues found that 70 percent had acknowledged, anonymously, to cutting some corners in reporting data. About a third said they had reported an unexpected finding as predicted from the start, and about 1 percent admitted to falsifying data.
  • Also common is a self-serving statistical sloppiness. In an analysis published this year, Dr. Wicherts and Marjan Bakker, also at the University of Amsterdam, searched a random sample of 281 psychology papers for statistical errors. They found that about half of the papers in high-end journals contained some statistical error, and that about 15 percent of all papers had at least one error tha
  • t changed a reported finding — almost always in opposition to the authors’ hypothesis.
  • an analysis of 49 studies appearing Wednesday in the journal PLoS One, by Dr. Wicherts, Dr. Bakker and Dylan Molenaar, found that the more reluctant that scientists were to share their data, the more likely that evidence contradicted their reported findings.
  • “We know the general tendency of humans to draw the conclusions they want to draw — there’s a different threshold,” said Joseph P. Simmons, a psychologist at the University of Pennsylvania’s Wharton School. “With findings we want to see, we ask, ‘Can I believe this?’ With those we don’t, we ask, ‘Must I believe this?’
peterconnelly

Facebook-parent Meta to share more details with researchers about political ad targetin... - 0 views

  • (CNN Business)Facebook-parent Meta on Monday said it would soon offer more transparency and information to researchers about how political and social ads are targeted to users on the platform, months before the US midterm elections.
  • The effort, dubbed the Facebook Open Research and Transparency project, was created to help qualified academic researchers study social media's impact on society with measures included to protect users' privacy, according to the company.
  • The data provided to researchers will include information such as the interest categories, which can include everything from "environmentalism" to "frequent travelers," chosen to help target each individual ad.
  • ...1 more annotation...
  • we hope to help people better understand the practices used to reach potential voters on our technologies
Javier E

Vitamins Hide the Low Quality of Our Food - NYTimes.com - 0 views

  • we fail to notice that food marketers use synthetic vitamins to sell unhealthful products. Not only have we become dependent on these synthetic vitamins to keep ourselves safe from deficiencies, but the eating habits they encourage are having disastrous consequences on our health.
  • vitamins spread from the labs of scientists to the offices of food marketers, and began to take on a life of their own.
  • Nutritionists are correct when they tell us that most of us don’t need to be taking multivitamins. But that’s only because multiple vitamins have already been added to our food.
  • ...11 more annotations...
  • Given the poor quality of the typical American diet, this fortification is far from superfluous. In fact, for products like milk and flour, where fortification and enrichment have occurred for so long that they’ve become invisible, it would be almost irresponsible not to add synthetic vitamins.
  • synthetic vitamins are as essential to food companies as they are to us. To be successful in today’s market, food manufacturers must create products that can be easily transported over long distances and stored for extended periods.
  • They also need to be sure that their products offer some nutritional value so that customers don’t have to go elsewhere to meet their vitamin needs. But the very processing that’s necessary to create long shelf lives destroys vitamins, among other important nutrients. It’s nearly impossible to create foods that can sit for months in a supermarket that are also naturally vitamin-rich.
  • Today, it would be easy to blame food marketers for using vitamins to deceive us into buying their products. But our blindness is largely our own fault.
  • we’ve entered into a complicit agreement with them: They depend on us to buy their products, and we depend on the synthetic vitamins they add to those products to support eating habits that might otherwise leave us deficient
  • extra vitamins do not protect us from the long-term “diseases of civilization” that are currently ravaging our country, including obesity, heart disease and Type 2 diabetes — many of which are strongly associated with diet.
  • natural foods contain potentially protective substances such as phytochemicals and polyunsaturated fat that also are affected by processing, but that are not usually replaced. If these turn out to be as important as many researchers suspect, then our exclusive focus on vitamins could mean we’re protecting ourselves against the wrong dangers. It’s as if we’re taking out earthquake insurance policies in an area more at risk for floods.
  • And adding back vitamins after the fact ignores the issue of synergy: how nutrients work naturally as opposed to when they are isolated. A 2011 study on broccoli, for example, found that giving subjects fresh broccoli florets led them to absorb and metabolize seven times more of the anticancer compounds known as glucosinolates, present in broccoli and other cruciferous vegetables
  • And yet we refuse to change our eating habits in the ways that would actually protect us, which would require refocusing our diets on minimally processed foods that are naturally nutrient-rich.
  • The popularity of dietary supplements and vitamin-enhanced processed “health” foods means that even those of us who try to do right by our health are often getting it wrong.
  • we mustn’t let it distract us from an even more fundamental question: how we’ve allowed the word “vitamin” to become synonymous with “health.”
Javier E

The Government's Bad Diet Advice - NYTimes.com - 0 views

  • How did experts get it so wrong?
  • the primary problem is that nutrition policy has long relied on a very weak kind of science: epidemiological, or “observational,” studies in which researchers follow large groups of people over many years. But even the most rigorous epidemiological studies suffer from a fundamental limitation. At best they can show only association, not causation.
  • Instead of accepting that this evidence was inadequate to give sound advice, strong-willed scientists overstated the significance of their studies.
  • ...7 more annotations...
  • Much of the epidemiological data underpinning the government’s dietary advice comes from studies run by Harvard’s school of public health. In 2011, directors of the National Institute of Statistical Sciences analyzed many of Harvard’s most important findings and found that they could not be reproduced in clinical trials.
  • In 2013, government advice to reduce salt intake (which remains in the current report) was contradicted by an authoritative Institute of Medicine study. And several recent meta-analyses have cast serious doubt on whether saturated fats are linked to heart disease, as the dietary guidelines continue to assert.
  • In clearing our plates of meat, eggs and cheese (fat and protein), we ate more grains, pasta and starchy vegetables (carbohydrates). Over the past 50 years, we cut fat intake by 25 percent and increased carbohydrates by more than 30 percent, according to a new analysis of government data. Yet recent science has increasingly shown that a high-carb diet rich in sugar and refined grains increases the risk of obesity, diabetes and heart disease — much more so than a diet high in fat and cholesterol.
  • Today, we are poised to make the same mistakes. The committee’s new report also advised eliminating “lean meat” from the list of recommended healthy foods, as well as cutting back on red and processed meats. Fewer protein choices will likely encourage Americans to eat even more carbs. It will also have policy implications: Meat could be limited in school lunches and other federal food programs.
  • It’s possible that a mostly meatless diet could be healthy for all Americans — but then again, it might not be. We simply do not know. There are no rigorous clinical trials on such a diet, and although epidemiological data exists for adult vegetarians, there is none for children.
  • We have to start looking more skeptically at epidemiological studies and rethinking nutrition policy from the ground up.
  • Until then, we would be wise to return to what worked better for previous generations: a diet that included fewer grains, less sugar and more animal foods like meat, full-fat dairy and eggs
Javier E

Vaccine Critics Turn Defensive Over Measles - NYTimes.com - 1 views

  • the parents at the heart of America’s anti-vaccine movement are being blamed for incubating an otherwise preventable public-health crisis.
  • officials scrambled to try to contain a wider spread of the highly contagious disease — which America declared vanquished 15 years ago, before a statistically significant number of parents started refusing to vaccinate their children.
  • The anti-vaccine movement can largely be traced to a 1998 report in a medical journal that suggested a link between vaccines and autism but was later proved fraudulent and retracted. Today, the waves of parents who shun vaccines include some who still believe in the link and some, like the Amish, who have religious objections to vaccines. Then there is a particular subculture of largely wealthy and well-educated families, many living in palmy enclaves around Los Angeles and San Francisco, who are trying to carve out “all-natural” lives for their children.
  • ...9 more annotations...
  • “Sometimes, I feel like we’re practicing in the 1950s,” said Dr. Eric Ball, a pediatrician in southern Orange County, where some schools report that 50 to 60 percent of their kindergartners are not fully vaccinated and that 20 to 40 percent of parents have sought a personal beliefs exemption to vaccination requirements. “It’s very frustrating. It’s hard to see a kid suffer for something that’s entirely preventable.”
  • Dr. Ball said he spent many days trying to persuade parents to vaccinate their children. He tries to alleviate their concerns. He shows parents his own children’s vaccine records. But it has not worked, and lately, as worries and anger over this outbreak have spread, some families who support vaccines have said they do not want to be in the same waiting room as unvaccinated families. The clinic where Dr. Ball works has treated unvaccinated children for years, but its staff is meeting next week to discuss a ban.“Our patients are really scared,” Dr. Ball said. “Our nightmare would be for someone to show up at our door with the measles.”
  • Norm Warren, the manager of the supermarket in Kearny, Gordon’s IGA, has changed his thinking toward those who do not vaccinate their children.“Before, I thought, ‘If you think your child will become autistic, fine.’ But now they’re pushing their beliefs on everybody, and I feel differently,“ he said. “How many lives have been saved by vaccination?“
  • Tobias has endured chickenpox and whooping cough, though Ms. McMenimen said the latter seemed more like a common cold. She considered a tetanus shot after he cut himself on a wire fence but decided against it: “He has such a strong immune system.”
  • In San Geronimo, Calif., a mostly rural community of rolling hills and oak trees about 30 miles north of San Francisco, 40 percent of the students walking into Lagunitas Elementary School have not been inoculated against measles, according to the school’s figures. Twenty-five percent have not been vaccinated for polio. In all, the state says that 58 percent of Lagunitas kindergartners do not have up-to-date vaccine records.
  • “A lot of people here have personal beliefs that are faith based,” said John Carroll, the school superintendent, who sent a letter home to parents last week encouraging them to vaccinate their children. The faith, Mr. Carroll said, is not so much religious as it is a belief that “they raise their children in a natural, organic environment” and are suspicious of pharmaceutical companies and big business.
  • Some parents forgo shots altogether. Others split vaccine doses or stretch out their timeline, worried about somehow overwhelming their children’s immune system. Kelly McMenimen, a Lagunitas parent, said she “meditated on it a lot” before deciding not to vaccinate her son Tobias, 8, against even “deadly or deforming diseases.” She said she did not want “so many toxins” entering the slender body of a bright-eyed boy who loves math and geography.
  • Members of the anti-vaccine movement said the public backlash had terrified many parents. “People are now afraid they’re going to be jailed,” said Barbara Loe Fisher, the president of the National Vaccine Information Center, a clearinghouse for resisters. “I can’t believe what I’m seeing. It’s gotten so out of hand, and it’s gotten so vicious.”
  • “It’s good to explore alternatives rather than go with the panic of everyone around you,” she said. “Vaccines don’t feel right for me and my family.”
grayton downing

Lost in Translation | The Scientist Magazine® - 0 views

  • excessive reporting of positive results in papers that describe animal testing of potential therapies, just like the publishing bias seen in clinical research, according to a paper published today
  • We know publication bias happens a lot in clinical trials,” said Torgerson, “so I was surprised at myself for being surprised at the results, because of course, if it happens in human research, why wouldn’t it happen in animal research?”
  • his suspicions about publication bias by performing a statistical meta-analysis of thousands of reported animal tests for various neurological interventions—a total of 4,445 reported tests of 160 different drugs and other treatments for conditions that included Alzheimer’s disease, Parkinson’s disease, brain ischemia, and more.
  • ...3 more annotations...
  • One, as mentioned, is the suppression of negative results. The second is selective reporting of only the statistical analyses of data that provide a significant score. “Practically any data set, if it is tortured enough, will confess, and you will get a statistically significant result,”
  • , a publication bias “almost certainly” applies in other areas of preclinical research,
  • he suggested that a forum where investigators can deposit neutral or negative results in the form of articles, should be established to ensure that such findings are in the public arena and not hidden. This should help prevent other researchers from pursuing fruitless avenues of research, he said, “which is a waste of animals and a waste of research money.” Not to mention a risk to people enrolled in potentially pointless trials.
Javier E

There's No Such Thing As 'Sound Science' | FiveThirtyEight - 1 views

  • cience is being turned against itself. For decades, its twin ideals of transparency and rigor have been weaponized by those who disagree with results produced by the scientific method. Under the Trump administration, that fight has ramped up again.
  • The same entreaties crop up again and again: We need to root out conflicts. We need more precise evidence. What makes these arguments so powerful is that they sound quite similar to the points raised by proponents of a very different call for change that’s coming from within science.
  • Despite having dissimilar goals, the two forces espouse principles that look surprisingly alike: Science needs to be transparent. Results and methods should be openly shared so that outside researchers can independently reproduce and validate them. The methods used to collect and analyze data should be rigorous and clear, and conclusions must be supported by evidence.
  • ...26 more annotations...
  • they’re also used as talking points by politicians who are working to make it more difficult for the EPA and other federal agencies to use science in their regulatory decision-making, under the guise of basing policy on “sound science.” Science’s virtues are being wielded against it.
  • What distinguishes the two calls for transparency is intent: Whereas the “open science” movement aims to make science more reliable, reproducible and robust, proponents of “sound science” have historically worked to amplify uncertainty, create doubt and undermine scientific discoveries that threaten their interests.
  • “Our criticisms are founded in a confidence in science,” said Steven Goodman, co-director of the Meta-Research Innovation Center at Stanford and a proponent of open science. “That’s a fundamental difference — we’re critiquing science to make it better. Others are critiquing it to devalue the approach itself.”
  • alls to base public policy on “sound science” seem unassailable if you don’t know the term’s history. The phrase was adopted by the tobacco industry in the 1990s to counteract mounting evidence linking secondhand smoke to cancer.
  • The sound science tactic exploits a fundamental feature of the scientific process: Science does not produce absolute certainty. Contrary to how it’s sometimes represented to the public, science is not a magic wand that turns everything it touches to truth. Instead, it’s a process of uncertainty reduction, much like a game of 20 Questions.
  • Any given study can rarely answer more than one question at a time, and each study usually raises a bunch of new questions in the process of answering old ones. “Science is a process rather than an answer,” said psychologist Alison Ledgerwood of the University of California, Davis. Every answer is provisional and subject to change in the face of new evidence. It’s not entirely correct to say that “this study proves this fact,” Ledgerwood said. “We should be talking instead about how science increases or decreases our confidence in something.”
  • While insisting that they merely wanted to ensure that public policy was based on sound science, tobacco companies defined the term in a way that ensured that no science could ever be sound enough. The only sound science was certain science, which is an impossible standard to achieve.
  • “Doubt is our product,” wrote one employee of the Brown & Williamson tobacco company in a 1969 internal memo. The note went on to say that doubt “is the best means of competing with the ‘body of fact’” and “establishing a controversy.” These strategies for undermining inconvenient science were so effective that they’ve served as a sort of playbook for industry interests ever since
  • Doubt merchants aren’t pushing for knowledge, they’re practicing what Proctor has dubbed “agnogenesis” — the intentional manufacture of ignorance. This ignorance isn’t simply the absence of knowing something; it’s a lack of comprehension deliberately created by agents who don’t want you to know,
  • In the hands of doubt-makers, transparency becomes a rhetorical move. “It’s really difficult as a scientist or policy maker to make a stand against transparency and openness, because well, who would be against it?
  • But at the same time, “you can couch everything in the language of transparency and it becomes a powerful weapon.” For instance, when the EPA was preparing to set new limits on particulate pollution in the 1990s, industry groups pushed back against the research and demanded access to primary data (including records that researchers had promised participants would remain confidential) and a reanalysis of the evidence. Their calls succeeded and a new analysis was performed. The reanalysis essentially confirmed the original conclusions, but the process of conducting it delayed the implementation of regulations and cost researchers time and money.
  • Delay is a time-tested strategy. “Gridlock is the greatest friend a global warming skeptic has,” said Marc Morano, a prominent critic of global warming research
  • which has received funding from the oil and gas industry. “We’re the negative force. We’re just trying to stop stuff.”
  • these ploys are getting a fresh boost from Congress. The Data Quality Act (also known as the Information Quality Act) was reportedly written by an industry lobbyist and quietly passed as part of an appropriations bill in 2000. The rule mandates that federal agencies ensure the “quality, objectivity, utility, and integrity of information” that they disseminate, though it does little to define what these terms mean. The law also provides a mechanism for citizens and groups to challenge information that they deem inaccurate, including science that they disagree with. “It was passed in this very quiet way with no explicit debate about it — that should tell you a lot about the real goals,” Levy said.
  • in the 20 months following its implementation, the act was repeatedly used by industry groups to push back against proposed regulations and bog down the decision-making process. Instead of deploying transparency as a fundamental principle that applies to all science, these interests have used transparency as a weapon to attack very particular findings that they would like to eradicate.
  • Now Congress is considering another way to legislate how science is used. The Honest Act, a bill sponsored by Rep. Lamar Smith of Texas,3The bill has been passed by the House but still awaits a vote in the Senate. is another example of what Levy calls a “Trojan horse” law that uses the language of transparency as a cover to achieve other political goals. Smith’s legislation would severely limit the kind of evidence the EPA could use for decision-making. Only studies whose raw data and computer codes were publicly available would be allowed for consideration.
  • It might seem like an easy task to sort good science from bad, but in reality it’s not so simple. “There’s a misplaced idea that we can definitively distinguish the good from the not-good science, but it’s all a matter of degree,” said Brian Nosek, executive director of the Center for Open Science. “There is no perfect study.” Requiring regulators to wait until they have (nonexistent) perfect evidence is essentially “a way of saying, ‘We don’t want to use evidence for our decision-making,’
  • ost scientific controversies aren’t about science at all, and once the sides are drawn, more data is unlikely to bring opponents into agreement.
  • objective knowledge is not enough to resolve environmental controversies. “While these controversies may appear on the surface to rest on disputed questions of fact, beneath often reside differing positions of value; values that can give shape to differing understandings of what ‘the facts’ are.” What’s needed in these cases isn’t more or better science, but mechanisms to bring those hidden values to the forefront of the discussion so that they can be debated transparently. “As long as we continue down this unabashedly naive road about what science is, and what it is capable of doing, we will continue to fail to reach any sort of meaningful consensus on these matters,”
  • The dispute over tobacco was never about the science of cigarettes’ link to cancer. It was about whether companies have the right to sell dangerous products and, if so, what obligations they have to the consumers who purchased them.
  • Similarly, the debate over climate change isn’t about whether our planet is heating, but about how much responsibility each country and person bears for stopping it
  • While researching her book “Merchants of Doubt,” science historian Naomi Oreskes found that some of the same people who were defending the tobacco industry as scientific experts were also receiving industry money to deny the role of human activity in global warming. What these issues had in common, she realized, was that they all involved the need for government action. “None of this is about the science. All of this is a political debate about the role of government,”
  • These controversies are really about values, not scientific facts, and acknowledging that would allow us to have more truthful and productive debates. What would that look like in practice? Instead of cherry-picking evidence to support a particular view (and insisting that the science points to a desired action), the various sides could lay out the values they are using to assess the evidence.
  • For instance, in Europe, many decisions are guided by the precautionary principle — a system that values caution in the face of uncertainty and says that when the risks are unclear, it should be up to industries to show that their products and processes are not harmful, rather than requiring the government to prove that they are harmful before they can be regulated. By contrast, U.S. agencies tend to wait for strong evidence of harm before issuing regulations
  • the difference between them comes down to priorities: Is it better to exercise caution at the risk of burdening companies and perhaps the economy, or is it more important to avoid potential economic downsides even if it means that sometimes a harmful product or industrial process goes unregulated?
  • But science can’t tell us how risky is too risky to allow products like cigarettes or potentially harmful pesticides to be sold — those are value judgements that only humans can make.
sissij

Tesla Passes Ford in Market Value as Investors Bet on the Future - The New York Times - 0 views

  • But there is one exception. Tesla, the electric-vehicle upstart, continues to surge.
  • “Investors want something that is going to go up in orders of magnitude in six months to six years, and Tesla is that story,” said Karl Brauer, a senior editor at Kelley Blue Book. “Nobody thinks Ford or G.M. is going to do that.”
  • Tesla’s chief executive, Elon Musk, has shattered the conventional wisdom that automakers should be viewed as a stable, reliable investment. Instead, he promotes his California-based company as a dynamic vehicle for growth, despite the risks and challenges ahead of it.
  • ...3 more annotations...
  • But neither automaker has convinced Wall Street that it has shed its boom-or-bust reputation tied to broader economic cycles, or is at the forefront of new technology being developed for self-driving vehicles and electric cars.
  • “It’s almost like Tesla is positioned in people’s minds as an energy storage company that happens to put most of its batteries on wheels,” said Andrew Stewart, chief investment officer at Exchange Capital Management, an investment firm in Ann Arbor, Mich.
  • While Tesla may enjoy the favor of investors, it still faces some daunting hurdles to reach its goals.
  •  
    In my research on Tesla, I found it very interesting that Tesla never has advertisement spreading out like Ford, Motor or other motor companies do. Yet, it is very popular and well-known. How does Tesla manage to be known by the public if they don't have any advertisement and their target costumers are the elites? One of the reason I found online about its propaganda strategy is its skill on giving stock holder confidence. Thus their stock price is always positive and healthy. By generating new ideas, Tesla is able to stay on the headline of the newspaper. When I saw this news, my first reaction is that it's Tesla again and give me a very positive image on the future of Tesla. This new way of propaganda is directly related to the new form of economics in the society so I found it very interesting. --Sissi (4/4/2017)
Javier E

Measles Proves Delicate Issue to G.O.P. Field - NYTimes.com - 0 views

  • The politics of medicine, morality and free will have collided in an emotional debate over vaccines and the government’s place in requiring them, posing a challenge for Republicans who find themselves in the familiar but uncomfortable position of reconciling modern science with the skepticism of their core conservative voters.
  • the national debate is forcing the Republican Party’s 2016 presidential hopefuls to confront questions about whether it is in the public’s interest to allow parents to decide for themselves.
  • The vaccination controversy is a twist on an old problem for the Republican Party: how to approach matters that have largely been settled among scientists but are not widely accepted by conservatives.
  • ...6 more annotations...
  • It is a dance Republican candidates often do when they hedge their answers about whether evolution should be taught in schools. It is what makes the fight over global warming such a liability for their party, and what led last year to a widely criticized response to the Ebola scare.
  • There is evidence that vaccinations have become more of a political issue in recent years. Pew Research Center polls show that in 2009, 71 percent of both Republicans and Democrats favored requiring the vaccination of children. Five years later, Democratic support had grown to 76 percent, but Republican support had fallen to 65 percent.
  • The debate does not break entirely along right-left lines. The movement to forgo vaccinations has been popular in more liberal and affluent communities where some parents are worried that vaccines cause autism or other disorders among children.
  • Howard Dean, a presidential candidate in 2004 and a former chairman of the Democratic National Committee, said there are three groups of people who object to required vaccines: “One is people who are very much scared about their kids getting autism, which is an idea that has been completely discredited. Two, is entitled people who don’t want to put any poison in their kids and view this as poison, which is ignorance more than anything else. And three, people who are antigovernment in any way.”
  • The issue has more political potency among conservative voters who are highly skeptical of anything required by the government.
  • for Republicans like Mr. Paul who appeal to the kind of libertarian conservatives who are influential in states like Iowa and New Hampshire, which hold the first two contests in the battle for the nomination, there is an appeal in framing the issue as one of individual liberty.Asked about immunizations again later on Monday, Mr. Paul was even more insistent, saying it was a question of “freedom.” He grew irritated with a CNBC host who pressed him and snapped: “The state doesn’t own your children. Parents own the children.”
dpittenger

Departing Leader of CERN Ponders Uncertainties That Lie Ahead - 0 views

  • Dr. Heuer, born in Bad Boll in southern Germany in 1948, has spent his career in the trenches of particle physics, in which scientists emulate 3-year-olds by smashing bits of matter together to see what comes out.
  • He had an opportunity to put that philosophy to the test early in his term at CERN, when physicists reported in a seminar there that they had measured subatomic particles known as neutrinos streaming from Geneva to their detector in Italy faster than the speed of light, contrary to the laws of physics then known.
  • The neutrino controversy helped set a sort of dubious stage for the main event in particle physics so far this century: the Higgs boson.
  • ...2 more annotations...
  • The Higgs boson completed the Standard Model, a suite of equations that agrees with all the experiments that have been done on earth. But that model is not the end of physics. It does not explain dark matter or dark energy, the two major constituents of the cosmos, for example, or why the universe is made of matter instead of antimatter.
  • For decades, theorists have flirted with a concept called supersymmetry that would address some of these issues and produce a bounty of new particles for CERN’s collider.
Javier E

Deeper Ties to Corporate Cash for Doubtful Climate Researcher - NYTimes.com - 1 views

  • For years, politicians wanting to block legislation on climate change have bolstered their arguments by pointing to the work of a handful of scientists who claim that greenhouse gases pose little risk to humanity.
  • One of the names they invoke most often is Wei-Hock Soon, known as Willie, a scientist at the Harvard-Smithsonian Center for Astrophysics who claims that variations in the sun’s energy can largely explain recent global warming.
  • He has accepted more than $1.2 million in money from the fossil-fuel industry over the last decade while failing to disclose that conflict of interest in most of his scientific papers. At least 11 papers he has published since 2008 omitted such a disclosure, and in at least eight of those cases, he appears to have violated ethical guidelines of the journals that published his work.
  • ...11 more annotations...
  • Historians and sociologists of science say that since the tobacco wars of the 1960s, corporations trying to block legislation that hurts their interests have employed a strategy of creating the appearance of scientific doubt, usually with the help of ostensibly independent researchers who accept industry funding.
  • “The whole doubt-mongering strategy relies on creating the impression of scientific debate,” said Naomi Oreskes, a historian of science at Harvard University and the co-author of “Merchants of Doubt,” a book about such campaigns. “Willie Soon is playing a role in a certain kind of political theater.”
  • Environmentalists have long questioned Dr. Soon’s work, and his acceptance of funding from the fossil-fuel industry was previously known. But the full extent of the links was not; the documents show that corporate contributions were tied to specific papers and were not disclosed, as required by modern standards of publishing.
  • “What it shows is the continuation of a long-term campaign by specific fossil-fuel companies and interests to undermine the scientific consensus on climate change,” said Kert Davies, executive director of the Climate Investigations Center, a group funded by foundations seeking to limit the risks of climate change.
  • Many experts in the field say that Dr. Soon uses out-of-date data, publishes spurious correlations between solar output and climate indicators, and does not take account of the evidence implicating emissions from human behavior in climate change.
  • Though often described on conservative news programs as a “Harvard astrophysicist,” Dr. Soon is not an astrophysicist and has never been employed by Harvard. He is a part-time employee of the Smithsonian Institution with a doctoral degree in aerospace engineering. He has received little federal research money over the past decade and is thus responsible for bringing in his own funds, including his salary.
  • Though he has little formal training in climatology, Dr. Soon has for years published papers trying to show that variations in the sun’s energy can explain most recent global warming. His thesis is that human activity has played a relatively small role in causing climate change.
  • As the oil-industry contributions fell, Dr. Soon started receiving hundreds of thousands of dollars through DonorsTrust, an organization based in Alexandria, Va., that accepts money from donors who wish to remain anonymous, then funnels it to various conservative causes.
  • Gavin A. Schmidt, head of the Goddard Institute for Space Studies in Manhattan, a NASA division that studies climate change, said that the sun had probably accounted for no more than 10 percent of recent global warming and that greenhouse gases produced by human activity explained most of it.“The science that Willie Soon does is almost pointless,” Dr. Schmidt said.
  • Dr. Soon has found a warm welcome among politicians in Washington and state capitals who try to block climate action. United States Senator James M. Inhofe, an Oklahoma Republican who claims that climate change is a global scientific hoax, has repeatedly cited Dr. Soon’s work over the years.
  • Dr. Oreskes, the Harvard science historian, said that academic institutions and scientific journals had been too lax in recent decades in ferreting out dubious research created to serve a corporate agenda.
sissij

New Prospects for Growing Human Replacement Organs in Animals - The New York Times - 0 views

  • For the first time, biologists have succeeded in growing human stem cells in pig embryos, shifting from science fiction to the realm of the possible the idea of developing human organs in animals for later transplant.
  • Since the organ would be made of a patient’s own cells, there would be little risk of immune rejection.
  • They would be generated by implanting human stem cells into an early pig embryo, resulting in an animal composed of mixed pig and human cells.
  • ...5 more annotations...
  • The two reports together establish the feasibility of trying to grow replacement human organs in animals, though such a goal is still far off.
  • Creating chimeras, especially those with human cells, may prove controversial, given the possibility that test animals could be humanized in undesirable ways. One would be if human cells should be incorporated into a pig’s brain, endowing it with human qualities. Almost no one wants a talking pig.
  • The ban is still in place, and it’s unclear whether the Trump administration would continue to consider lifting the moratorium or whether new objections would be raised to using public funds for this line of research.
  • But no one knows exactly what sequence of chemicals is required for the generation of each different tissue or organ. This may be why glassware experiments with stem cells have not yet lived up to their full promise.
  • Concern about human cells’ incorporation into a lower animal’s brain is not without basis. Dr. Steven Goldman of the University of Rochester Medical Center found in 2013 that mice injected with a special type of human brain cell had enhanced learning abilities.
  •  
    The ethics in biology is always a controversial issue in the research. In biology, we want to get as close to the truth as we possibly can, but that sometimes means that we have to research on things that are considered to be inhumane. This article talks about a new possibility that we can grow human stem cells in animals. That would lead to a potential problem of animals gaining intelligence. If animals start to have human properties, how would we treat them? Will they be a threat to our identity? And also, through the experiment described in the article, we can see that the scientific method in biology is dealing with probabilism and population. It is always gathering data. They are always ready for exceptions. --Sissi (1/28/2017)
1 - 20 of 85 Next › Last »
Showing 20 items per page