Group items tagged

science has long been married to mathematics. Generally it has been for the better. Especially since the days of Galileo and Newton, math has nurtured science. Rigorous mathematical methods have secured science’s fidelity to fact and conferred a timeless reliability to its findings.

a mutant form of math has deflected science’s heart from the modes of calculation that had long served so faithfully. Science was seduced by statistics, the math rooted in the same principles that guarantee profits for Las Vegas casinos. Supposedly, the proper use of statistics makes relying on scientific results a safe bet. But in practice, widespread misuse of statistical methods makes science more like a crapshoot.

science’s dirtiest secret: The “scientific method” of testing hypotheses by statistical analysis stands on a flimsy foundation. Statistical tests are supposed to guide scientists in judging whether an experimental result reflects some real effect or is merely a random fluke, but the standard methods mix mutually inconsistent philosophies and offer no meaningful basis for making such decisions. Even when performed correctly, statistical tests are widely misunderstood and frequently misinterpreted. As a result, countless conclusions in the scientific literature are erroneous, and tests of medical dangers or treatments are often contradictory and confusing.

On September 18, 2007, a few dozen neuroscientists, psychiatrists, and drug-company executives gathered in a hotel conference room in Brussels to hear some startling news. It had to do with a class of drugs known as atypical or second-generation antipsychotics, which came on the market in the early nineties.

the therapeutic power of the drugs appeared to be steadily waning. A recent study showed an effect that was less than half of that documented in the first trials, in the early nineteen-nineties. Many researchers began to argue that the expensive pharmaceuticals weren’t any better than first-generation antipsychotics, which have been in use since the fifties. “In fact, sometimes they now look even worse,” John Davis, a professor of psychiatry at the University of Illinois at Chicago, told me.

Whether individual Likert items can be considered as interval-level data, or whether they should be considered merely ordered-categorical data is the subject of disagreement. Many regard such items only as ordinal data, because, especially when using only five levels, one cannot assume that respondents perceive all pairs of adjacent levels as equidistant. On the other hand, often (as in the example above) the wording of response levels clearly implies a symmetry of response levels about a middle category; at the very least, such an item would fall between ordinal- and interval-level measurement; to treat it as merely ordinal would lose information. Further, if the item is accompanied by a visual analog scale, where equal spacing of response levels is clearly indicated, the argument for treating it as interval-level data is even stronger.

When treated as ordinal data, Likert responses can be collated into bar charts, central tendency summarised by the median or the mode (but some would say not the mean), dispersion summarised by the range across quartiles (but some would say not the standard deviation), or analyzed using non-parametric tests, e.g. chi-square test, Mann–Whitney test, Wilcoxon signed-rank test, or Kruskal–Wallis test.[4] Parametric analysis of ordinary averages of Likert scale data is also justifiable by the Central Limit Theorem, although some would disagree that ordinary averages should be used for Likert scale data.

The truth is not all screening tests are equal, and more importantly, even when a screening test is accurate (yes, I chose to use this term because I am lazy) and we are able "determine the health condition of an individual", it did not necessarily mean that it was cost-effective to screen the population at large, or indeed a specific patient. Unfortunately, if someone was in a position that "the costs of these tests are a big burden" to him or her, an accurate diagnosis may just be the beginning of more financial burden...

But the cost-effectiveness of screening tests are really quite a technical issue that we cannot expect laymen (or even all doctors) to understand - that's not what bothered me about this letter. What bothered me was how a layman can think that a bunch of doctors and statisticians sitting in MOH can be blind to the benefits of "all cancer screenings as well as all tests to detect heart disease", which are so obvious to him.

One of the big misconceptions is that artificial chemicals such as pesticides have a lot to do with human cancer, but that's just not true," says Bruce N. Ames, professor of biochemistry and molecular biology at the University of California at Berkeley and co-author of a new review of what is known about environmental pollution and cancer. "Nevertheless, it's conventional wisdom and society spends billions on this each year." "We consume more carcinogens in one cup of coffee than we get from the pesticide residues on all the fruits and vegetables we eat in a year," he adds.

there may be many excellent reasons for cleaning up pollution of our air, water and soil, the researchers say, prevention of cancer is not one of them.

"The problem is that lifestyle changes are tough," says Gold, director of the Carcinogenic Potency Project at UC Berkeley's National Institute for Environmental Health Sciences Center and a senior scientist in the cell and molecular biology division at Lawrence Berkeley National Laboratory. "But by targeting pesticide residues as a major problem, we risk making fruits and vegetables more expensive and indirectly increasing cancer risks, especially among the poor."

a New York Times article covered the stiff competition for entrance to public and private kindergartens in Manhattan for especially smart kids.  Whereas at one time teachers recommended students to these programs, today entrance to both public and private schools for gifted children is dependent entirely on test scores.

It’s unfair that entrance into kindergarten level programs is being gamed by people with resources, disadvantaging the most disadvantaged kids from the get go.  I think it’s egregious.  Many people will agree that this isn’t fair.

But the more insidious value, the one that almost no one would identify as problematic, is the idea that all parents should do everything they can to give their child advantages.  Even Ms. Stewart thinks so.  “They want to help their kids,” she said. “If I could buy it, I would, too.” Somehow, in the attachment to the idea that we should all help our kids get every advantage, the fact that advantaging your child disadvantages other people’s children gets lost.  If it advantages your child, it must be advantaging him over someone else; otherwise it’s not an advantage, you see?

Scientists these days tend to keep up a polite fiction that all science is equal. Except for the work of the misguided opponent whose arguments we happen to be refuting at the time, we speak as though every scientist's field and methods of study are as good as every other scientist's and perhaps a little better. This keeps us all cordial when it comes to recommending each other for government grants.

Why should there be such rapid advances in some fields and not in others? I think the usual explanations that we tend to think of - such as the tractability of the subject, or the quality or education of the men drawn into it, or the size of research contracts - are important but inadequate. I have begun to believe that the primary factor in scientific advance is an intellectual one. These rapidly moving fields are fields where a particular method of doing scientific research is systematically used and taught, an accumulative method of inductive inference that is so effective that I think it should be given the name of "strong inference." I believe it is important to examine this method, its use and history and rationale, and to see whether other groups and individuals might learn to adopt it profitably in their own scientific and intellectual work. In its separate elements, strong inference is just the simple and old-fashioned method of inductive inference that goes back to Francis Bacon. The steps are familiar to every college student and are practiced, off and on, by every scientist. The difference comes in their systematic application. Strong inference consists of applying the following steps to every problem in science, formally and explicitly and regularly: Devising alternative hypotheses; Devising a crucial experiment (or several of them), with alternative possible outcomes, each of which will, as nearly is possible, exclude one or more of the hypotheses; Carrying out the experiment so as to get a clean result; Recycling the procedure, making subhypotheses or sequential hypotheses to refine the possibilities that remain, and so on.

On any new problem, of course, inductive inference is not as simple and certain as deduction, because it involves reaching out into the unknown. Steps 1 and 2 require intellectual inventions, which must be cleverly chosen so that hypothesis, experiment, outcome, and exclusion will be related in a rigorous syllogism; and the question of how to generate such inventions is one which has been extensively discussed elsewhere (2, 3). What the formal schema reminds us to do is to try to make these inventions, to take the next step, to proceed to the next fork, without dawdling or getting tied up in irrelevancies.

Do the languages we speak shape the way we see the world, the way we think, and the way we live our lives? Do people who speak different languages think differently simply because they speak different languages? Does learning new languages change the way you think? Do polyglots think differently when speaking different languages?

For a long time, the idea that language might shape thought was considered at best untestable and more often simply wrong. Research in my labs at Stanford University and at MIT has helped reopen this question. We have collected data around the world: from China, Greece, Chile, Indonesia, Russia, and Aboriginal Australia.

What we have learned is that people who speak different languages do indeed think differently and that even flukes of grammar can profoundly affect how we see the world.

Columnist David Leonhardt reports the findings of a new study which suggests that children who are fortunate enough to have an unusually good kindergarten teacher can expect to make roughly an extra twenty dollars a week by the age of twenty-seven.

it implies that during each school year a good kindergarten teacher creates an additional $320,000 of earnings.

The new research (pdf), the work of six economists—four from Harvard, one from Berkeley, and one from Northwestern—upends this finding. It is based on test scores and demographic data from a famous experiment carried out in Tennessee during the late nineteen-eighties, which tracked the progress of about 11,500 students from kindergarten to third grade. Most of these students are now about thirty years old, which means they have been working for up to twelve years. The researchers also gained access to income-tax data and matched it up with the test scores. Their surprising conclusion is that the uplifting effect of a good kindergarten experience, after largely disappearing during a child’s teen years, somehow reappears in the adult workplace. (See Figure 7 in the paper.) Why does this happen? The author don’t say, but Leonhardt offers this explanation: “Good early education can impart skills that last a lifetime—patience, discipline, manners, perseverance. The tests that 5-year-olds take may pick up these skills, even if later multiple-choice tests do not.”

Question #1: Does this mean I don’t have to believe in climate change? Me: I’m afraid not. One of the sad ironies of scientific denialism is that we tend to be skeptical of precisely the wrong kind of scientific claims. In poll after poll, Americans have dismissed two of the most robust and widely tested theories of modern science: evolution by natural selection and climate change. These are theories that have been verified in thousands of different ways by thousands of different scientists working in many different fields. (This doesn’t mean, of course, that such theories won’t change or get modified – the strength of science is that nothing is settled.) Instead of wasting public debate on creationism or the rhetoric of Senator Inhofe, I wish we’d spend more time considering the value of spinal fusion surgery, or second generation antipsychotics, or the verity of the latest gene association study. The larger point is that we need to be a better job of considering the context behind every claim. In 1952, the Harvard philosopher Willard Von Orman published “The Two Dogmas of Empiricism.” In the essay, Quine compared the truths of science to a spider’s web, in which the strength of the lattice depends upon its interconnectedness. (Quine: “The unit of empirical significance is the whole of science.”) One of the implications of Quine’s paper is that, when evaluating the power of a given study, we need to also consider the other studies and untested assumptions that it depends upon. Don’t just fixate on the effect size – look at the web. Unfortunately for the denialists, climate change and natural selection have very sturdy webs.

biases are not fraud. We sometimes forget that science is a human pursuit, mingled with all of our flaws and failings. (Perhaps that explains why an episode like Climategate gets so much attention.) If there’s a single theme that runs through the article it’s that finding the truth is really hard. It’s hard because reality is complicated, shaped by a surreal excess of variables. But it’s also hard because scientists aren’t robots: the act of observation is simultaneously an act of interpretation.

(As Paul Simon sang, “A man sees what he wants to see and disregards the rest.”) Most of the time, these distortions are unconscious – we don’t know even we are misperceiving the data. However, even when the distortion is intentional it’s still rarely rises to the level of outright fraud. Consider the story of Mike Rossner. He’s executive director of the Rockefeller University Press, and helps oversee several scientific publications, including The Journal of Cell Biology.  In 2002, while trying to format a scientific image in Photoshop that was going to appear in one of the journals, Rossner noticed that the background of the image contained distinct intensities of pixels. “That’s a hallmark of image manipulation,” Rossner told me. “It means the scientist has gone in and deliberately changed what the data looks like. What’s disturbing is just how easy this is to do.” This led Rossner and his colleagues to begin analyzing every image in every accepted paper. They soon discovered that approximately 25 percent of all papers contained at least one “inappropriately manipulated” picture. Interestingly, the vast, vast majority of these manipulations (~99 percent) didn’t affect the interpretation of the results. Instead, the scientists seemed to be photoshopping the pictures for aesthetic reasons: perhaps a line on a gel was erased, or a background blur was deleted, or the contrast was exaggerated. In other words, they wanted to publish pretty images. That’s a perfectly understandable desire, but it gets problematic when that same basic instinct – we want our data to be neat, our pictures to be clean, our charts to be clear – is transposed across the entire scientific process.

an interesting article I read recently, penned by Julian Savulescu for the Practical Ethics blog.

Savulescu discusses an ongoing controversy in Germany about genetic testing of human embryos. The Leopoldina, Germany’s equivalent of the National Academy of Sciences, has recommended genetic testing of pre-implant embryos, to screen for serious and incurable defects. The German Chancellor, Angela Merkel, has agreed to allow a parliamentary vote on this issue, but also said that she personally supports a ban on this type of testing. Her fear is that the testing would quickly lead to “designer babies,” i.e. to parents making choices about their unborn offspring based not on knowledge about serious disease, but simply because they happen to prefer a particular height or eye color.

He infers from Merkel’s comments (and many similar others) that people tend to think of selecting traits like eye color as eugenics, while acting to avoid incurable disease is not considered eugenics. He argues that this is exactly wrong: eugenics, as he points out, means “well born,” so eugenicists have historically been concerned with eliminating traits that would harm society (Wendell Holmes’ “three generation of imbeciles”), not with simple aesthetic choices. As Savulescu puts it: “[eugenics] is selecting embryos which are better, in this context, have better lives. Being healthy rather than sick is ‘better.’ Having blond hair and blue eyes is not in any plausible sense ‘better,’ even if people mistakenly think so.”

When approaching a problem, we can choose from any of several cognitive mechanisms. Some mechanisms have great computational power, letting us solve many problems with great accuracy, but they are slow, require much concentration and can interfere with other cognitive tasks. Others are comparatively low in computational power, but they are fast, require little concentration and do not interfere with other ongoing cognition. Humans are cognitive misers because our basic tendency is to default to the processing mechanisms that require less computational effort, even if they are less accurate.

our tendency to evaluate a situation from our own perspective. We weigh evidence and make moral judgments with a my-side bias that often leads to dysrationalia that is independent of measured intelligence. The same is true for other tendencies of the cognitive miser that have been much studied, such as attribute substitution and conjunction errors; they are at best only slightly related to intelligence and are poorly captured by conventional intelligence tests.

The USPSTF recently issued an even sharper warning about the prostate-specific antigen test for prostate cancer, after concluding that the test’s harms outweigh its benefits. Chest X-rays for lung cancer and Pap tests for cervical cancer have received similar, albeit less definitive, criticism.CommentsView/Create comment on this paragraphThe next step in the reevaluation of cancer screening was taken last year, when researchers at the Dartmouth Institute for Health Policy announced that the costs of screening for breast cancer were often minimized, and that the benefits were much exaggerated. Indeed, even a mammogram (almost 40 million are given annually in the US) that detects a cancer does not necessarily save a life.CommentsView/Create comment on this paragraphThe Dartmouth researchers found that, of the estimated 138,000 breast cancers detected annually in the US, the test did not help 120,000-134,000 of the afflicted women. The cancers either were growing so slowly that they did not pose a problem, or they would have been treated successfully if discovered clinically later (or they were so aggressive that little could be done).

MIDDLE SCHOOL students are champion time-wasters. And the personal computer may be the ultimate time-wasting appliance.

there is an automatic inclination to think of the machine in its most idealized form, as the Great Equalizer. In developing countries, computers are outfitted with grand educational hopes, like those that animate the One Laptop Per Child initiative, which was examined in this space in April.

Economists are trying to measure a home computer’s educational impact on schoolchildren in low-income households. Taking widely varying routes, they are arriving at similar conclusions: little or no educational benefit is found. Worse, computers seem to have further separated children in low-income households, whose test scores often decline after the machine arrives, from their more privileged counterparts.

ommentators have repeatedly told us in recent years that the gap between rich and poor has been widening. It is true, if you compare the income of those in the top fifth of earners with the income of those in the bottom fifth, that the spread between them increased between 1996 and 2005. But, as Sowell points out, this frequently cited figure is not counting the same people. If you look at individual taxpayers, Sowell notes, those who happened to be in the bottom fifth in 1996 saw their incomes nearly double over the decade, while those who happened to be in the top fifth in 1995 saw gains of only 10 percent on average and those in the top 5 percent actually experienced decline in their incomes. Similar distortions are perpetrated by those bewailing "stagnation" in average household incomes -- without taking into account that households have been getting smaller, as rising wealth allows people to move out of large family homes.

Sometimes the distortion seems to be deliberate. Sowell gives the example of an ABC news report in the 1980s focusing on five states where "unemployment is most severe" -- without mentioning that unemployment was actually declining in all the other 45 states. Sometimes there seems to be willful incomprehension. Journalists have earnestly reported that "prisons are ineffective" because two-thirds of prisoners are rearrested within three years of their release. As Sowell comments: "By this kind of reasoning, food is ineffective as a response to hunger because it is only a matter of time after eating before you get hungry again. Like many other things, incarceration only works when it is done."

why do intellectuals often seem so lacking in common sense? Sowell thinks it goes with the job-literally: He defines "intellectuals" as "an occupational category [Sowell's emphasis], people whose occupations deal primarily with ideas -- writers, academics and the like." Medical researchers or engineers or even "financial wizards" may apply specialized knowledge in ways that require great intellectual skill, but that does not make them "intellectuals," in Sowell's view: "An intellectual's work begins and ends with ideas [Sowell's emphasis]." So an engineer "is ruined" if his bridges or buildings collapse and so with a financier who "goes broke… the proof of the pudding is ultimately in the eating…. but the ultimate test of a [literary] deconstructionist's ideas is whether other deconstructionists find those ideas interesting, original, persuasive, elegant or ingenious. There is no external test." The ideas dispensed by intellectuals aren't subject to "external" checks or exposed to the test of "verifiability" (apart from what "like-minded individuals" find "plausible") and so intellectuals are not really "accountable" in the same way as people in other occupations.

Everyone is a skeptic nowadays, or so it seems. From climate change to evolution to vaccination, large proportions of the population claim to be skeptical about many of the claims of mainstream science. So why are we, member of the skeptical community, not rejoicing?

A skeptic, in popular discourse, is simply someone who denies a particular claim. But true skepticism, as espoused by philosophers and scientists for millenia, is more an intellectual attitude than a position on a specific issue. A skeptic is someone who always demands sufficient evidence or reasons before accepting a claim. This skeptical attitude – its opposite is credulity – leads skeptics to reject as unfounded any claim that cannot withstand the rigours of the scientific method, which includes controlled experimental testing. The more extraordinary the claim, the more rigourously it must be tested before a skeptic will be willing to accept

skepticism does not always lead to denial. Extraordinary claims require extraordinary evidence, but sometimes that extraordinary evidence can be provided. Einstein’s theory of relativity, which holds that matter can change the very shape of space and time, is an extraordinary claim, yet it has stood up to the most demanding of scientific testing.