Group items tagged

I was interested in seeing how geography and political borders affected where people lived relative to their friends. I wanted a visualization that would show which cities had a lot of friendships between them.

I began by taking a sample of about ten million pairs of friends from Apache Hive, our data warehouse. I combined that data with each user's current city and summed the number of friends between each pair of cities. Then I merged the data with the longitude and latitude of each city. At that point, I began exploring it in R, an open-source statistics environment. As a sanity check, I plotted points at some of the latitude and longitude coordinates. To my relief, what I saw was roughly an outline of the world. Next I erased the dots and plotted lines between the points. After a few minutes of rendering, a big white blob appeared in the center of the map. Some of the outer edges of the blob vaguely resembled the continents, but it was clear that I had too much data to get interesting results just by drawing lines. I thought that making the lines semi-transparent would do the trick, but I quickly realized that my graphing environment couldn't handle enough shades of color for it to work the way I wanted.

Instead I found a way to simulate the effect I wanted. I defined weights for each pair of cities as a function of the Euclidean distance between them and the number of friends between them. Then I plotted lines between the pairs by weight, so that pairs of cities with the most friendships between them were drawn on top of the others. I used a color ramp from black to blue to white, with each line's color depending on its weight. I also transformed some of the lines to wrap around the image, rather than spanning more than halfway around the world. View high-res (3.8MB)

In modern medicine, placebos are associated with another form of deception — a kind that has long been thought essential for conducting randomized clinical trials of new drugs, the statistical rock upon which the global pharmaceutical industry was built. One group of volunteers in an RCT gets the novel medication; another group (the “control” group) gets pills or capsules that look identical to the allegedly active drug, but contain only an inert substance like milk sugar. These faux drugs are called placebos.

Inevitably, the health of some people in both groups improves, while the health of others grows worse. Symptoms of illness fluctuate for all sorts of reasons, including regression to the mean.

Since the goal of an RCT, from Big Pharma’s perspective, is to demonstrate the effectiveness of a new drug, the return to robust health of a volunteer in the control group is considered a statistical distraction. If too many people in the trial get better after downing sugar pills, the real drug will look worse by comparison — sometimes fatally so for the purpose of earning approval from the Food and Drug Adminstration.

British sociologist Harry Collins asked a scientist who specializes in gravitational waves to answer seven questions about the physics of these waves. Collins, who has made an amateur study of this field for more than 30 years but has never actually practiced it, also answered the questions himself. Then he submitted both sets of answers to a panel of judges who are themselves gravitational-wave researchers. The judges couldn't tell the impostor from one of their own. Collins argues that he is therefore as qualified as anyone to discuss this field, even though he can't conduct experiments in it.

The journal Nature predicted that the experiment would have a broad impact, writing that Collins could help settle the "science wars of the 1990s," "when sociologists launched what scientists saw as attacks on the very nature of science, and scientists responded in kind," accusing the sociologists of misunderstanding science. More generally, it could affect "the argument about whether an outsider, such as an anthropologist, can properly understand another group, such as a remote rural community." With this comment, Nature seemed to be saying that if a sociologist can understand physics, then anyone can understand anything.

It will be interesting to see if Collins' results can indeed be repeated in different situations. Meanwhile, his experiment is plenty interesting in itself. Just one of the judges succeeded in distinguishing Collins' answers from those of the trained experts. One threw up his hands. And the other seven declared Collins the physicist. He didn't simply do as well as the trained specialist—he did better, even though the test questions demanded technical answers. One sample answer from Collins gives you the flavor: "Since gravitational waves change the shape of spacetime and radio waves do not, the effect on an interferometer of radio waves can only be to mimic the effects of a gravitational wave, not reproduce them." (More details can be found in this paper Collins wrote with his collaborators.)

Being comfortable with uncertainty, knowing the limits of what science can tell us, and understanding the worth of failure are all valuable tools that would improve people's lives, according to some of the world's leading thinkers.

he ideas were submitted as part of an annual exercise by the web magazine Edge, which invites scientists, philosophers and artists to opine on a major question of the moment. This year it was, "What scientific concept would improve everybody's cognitive toolkit?"

the public often misunderstands the scientific process and the nature of scientific doubt. This can fuel public rows over the significance of disagreements between scientists about controversial issues such as climate change and vaccine safety.

In your article, you make the argument that “Straits Times Forum Editor, was merely amending his (my) letter to cite the correct statistics. “For example, the Education Minister said “How children from the bottom one-third by socio-economic background fare: One in two scores in the top two-thirds at PSLE” - But, Mr Samuel Wee wrote “His statement is backed up with the statistic that 50% of children from the bottom third of the socio-economic ladder score in the bottom third of the Primary School Leaving Examination”.” Kind sir, the statistics state that 1 in 2 are in the top 66.6% (Which, incidentally, includes the top fifth of the bottom 50%!) Does it not stand to reason, then, that if 50% are in the top 66.6%, the remaining 50% are in the bottom 33.3%, as I stated in my letter?

Also, perhaps you were not aware of the existence of this resource, but here is a graph from the Straits Times illustrating the fact that only 10% of children from one-to-three room flats make it to university–which is to say, 90% of them don’t. http://www.straitstimes.com/STI/STIMEDIA/pdf/20110308/a10.pdf I look forward to your reply, Mr Leong. Thank you for taking the time to read this message.

we should, wherever possible, try to agree to disagree, as it is healthy to have and to encourage different viewpoints.

Prinz’s basic stance is that moral values stem from our cognitive hardware, upbringing, and social environment. These equip us with deep-seated moral emotions, but these emotions express themselves in a contingent way due to cultural circumstances. And while reason can help, it has limited influence, and can only reshape our ethics up to a point, it cannot settle major differences between different value systems. Therefore, it is difficult, if not impossible, to construct an objective morality that transcends emotions and circumstance.

As Prinz writes, in part:“No amount of reasoning can engender a moral value, because all values are, at bottom, emotional attitudes. … Reason cannot tell us which facts are morally good. Reason is evaluatively neutral. At best, reason can tell us which of our values are inconsistent, and which actions will lead to fulfillment of our goals. But, given an inconsistency, reason cannot tell us which of our conflicting values to drop or which goals to follow. If my goals come into conflict with your goals, reason tells me that I must either thwart your goals, or give up caring about mine; but reason cannot tell me to favor one choice over the other. … Moral judgments are based on emotions, and reasoning normally contributes only by helping us extrapolate from our basic values to novel cases. Reasoning can also lead us to discover that our basic values are culturally inculcated, and that might impel us to search for alternative values, but reason alone cannot tell us which values to adopt, nor can it instill new values.”

This moral relativism is not the absolute moral relativism of, supposedly, bands of liberal intellectuals, or of postmodernist philosophers. It presents a more serious challenge to those who argue there can be objective morality. To be sure, there is much Prinz and I agree on. At the least, we agree that morality is largely constructed by our cognition, upbringing, and social environment; and that reason has the power synthesize and clarify our worldviews, and help us plan for and react to life’s situations

he “weight of evidence” approach to evaluation of causality is often vilified by cell phone and WiFi scare mongers as being an inadequate way to judge the evidence – often because it disagrees with their own sentiments about the science.  If you can’t disqualify the evidence, then you can go after the method of evaluation and disqualify that, right?  Of course, the weight of evidence approach is often portrayed as a dumbshow of putting all the “positive” trials on one side of the scale and all of the “negative” trials on the other and taking the difference in mass as the evidence.  This is how Dr. Phillips characterised it in his paper on electromagnetic fields and DNA damage, as well as his appearance on CBC Radio.  Of course, the procedure is much more like a systematic review, where all of the papers, regardless of their outcomes, are weighed for their quality. (The higher quality studies will have good internal and external validity, proper blinding and randomisation, large enough sample size, proper controls and good statistical analysis; as well as being reproduced by independent investigators.) Then they are tallied and a rational conclusion is offered as to the most likely state of the evidence (of course, it is much more involved than I am stating, but suffice it to say, it does not involve a scale.)   This is standard operating procedure and, in fact, is what we all do when we are evaluating evidence: we decide which studies are good and we pool the evidence before we make a decision.

One of the scientists was Richard Muller from University of California, Berkeley. Muller has been working on an independent project to better estimate the planet's surface temperatures over time. Because he is willing to say publicly that he has some doubts about the accuracy of the temperature stations that most climate models are based on, he has been embraced by the science denying crowd.

A Koch brothers charity, for example, has donated nearly 25 percent of the financial support provided to Muller's project.

Skeptics of climate science have been licking their lips waiting for his latest research, which they hoped would undermine the data behind basic theories of anthropogenic climate change. At the hearing today, however, Muller threw them for a loop with this graph:

Lit Lab tackles literary problems by scientific means: hypothesis-testing, computational modeling, quantitative analysis. Similar efforts are currently proliferating under the broad rubric of “digital humanities,” but Moretti’s approach is among the more radical. He advocates what he terms “distant reading”: understanding literature not by studying particular texts, but by aggregating and analyzing massive amounts of data.

People recognize, say, Gothic literature based on castles, revenants, brooding atmospheres, and the greater frequency of words like “tremble” and “ruin.” Computers recognize Gothic literature based on the greater frequency of words like . . . “the.” Now, that’s interesting. It suggests that genres “possess distinctive features at every possible scale of analysis.” More important for the Lit Lab, it suggests that there are formal aspects of literature that people, unaided, cannot detect.

Distant reading might prove to be a powerful tool for studying literature, and I’m intrigued by some of the lab’s other projects, from analyzing the evolution of chapter breaks to quantifying the difference between Irish and English prose styles. But whatever’s happening in this paper is neither powerful nor distant. (The plot networks were assembled by hand; try doing that without reading Hamlet.) By the end, even Moretti concedes that things didn’t unfold as planned. Somewhere along the line, he writes, he “drifted from quantification to the qualitative analysis of plot.”