Everybody lies: how Google search reveals our darkest secrets | Technology | The Guardian - 0 views
shared by Javier E on 09 Jul 17
- No Cached
lasvegas-flights liked it
-
Many people underreport embarrassing behaviours and thoughts on surveys. They want to look good, even though most surveys are anonymous. This is called social desirability bias.
-
An important paper in 1950 provided powerful evidence of how surveys can fall victim to such bias. Researchers collected data, from official sources, on the residents of Denver: what percentage of them voted, gave to charity, and owned a library card. They then surveyed the residents to see if the percentages would match. The results were, at the time, shocking. What the residents reported to the surveys was very different from the data the researchers had gathered
-
Then there’s that odd habit we sometimes have of lying to ourselves. Lying to oneself may explain why so many people say they are above average. How big is this problem? More than 40% of one company’s engineers said they are in the top 5%. More than 90% of college professors say they do above-average work. One-quarter of high school seniors think they are in the top 1% in their ability to get along with other people. If you are deluding yourself, you can’t be honest in a survey.
- ...51 more annotations...
-
on sensitive topics, every survey method will elicit substantial misreporting. People have no incentive to tell surveys the truth.
-
How, therefore, can we learn what our fellow humans are really thinking and doing? Big data
-
Think of Google searches. Remember the conditions that make people more honest. Online? Check. Alone? Check. No person administering a survey? Check.
-
I am now convinced that Google searches are the most important dataset ever collected on the human psyche.
-
How many American men are gay? This is a regular question in sexuality research. Yet it has been among the toughest questions for social scientists to answer. Psychologists no longer believe Alfred Kinsey’s famous estimate – based on surveys that oversampled prisoners and prostitutes – that 10% of American men are gay. Representative surveys now tell us about 2% to 3% are
-
About 2.5% of male Facebook users who list a gender of interest say they are interested in men; that corresponds roughly with what the surveys indicate.
-
It turns out that wives suspect their husbands of being gay rather frequently. They demonstrate that suspicion in the surprisingly common search: “Is my husband gay?” The word “gay” is 10% more likely to complete searches that begin “Is my husband...” than the second-place word, “cheating”. It is eight times more common than “an alcoholic” and 10 times more common than “depressed”.
-
If mobility cannot fully explain why some states have so many more openly gay men, the closet must be playing a big role. Which brings us back to Google
-
about 5% of male porn searches are for gay-male porn. Overall, there are more gay porn searches in tolerant states compared with intolerant states.
-
one consequence of my estimate is clear: an awful lot of men in the United States, particularly in intolerant states, are still in the closet. They don’t reveal their sexual preferences on Facebook. They don’t admit it on surveys. And, in many cases, they may even be married to women.
-
There is clearly some mobility – from Oklahoma City to San Francisco, for example. But I estimate that men moving to someplace more open-minded can explain less than half of the difference in the openly gay population in tolerant versus intolerant states.
-
On Google, there are 16 times more complaints about a spouse not wanting sex than about a married partner not being willing to talk. There are five-and-a-half times more complaints about an unmarried partner not wanting sex than an unmarried partner refusing to text back.
-
Google data also suggests a reason people may be avoiding sex so frequently: enormous anxiety, with much of it misplaced. Start with men’s anxieties. It isn’t news that men worry about how well endowed they are, but the degree of this worry is rather profound. Men Google more questions about their sexual organ than any other body par
-
Men conduct more searches for how to make their penises bigger than how to tune a guitar, make an omelette, or change a tyre. Men’s top Googled concern about steroids isn’t whether they may damage their health but whether taking them might diminish the size of their penis. Men’s top Googled question related to how their body or mind would change as they aged was whether their penis would get smaller.
-
Do women care about penis size? Rarely, according to Google searches. For every search women make about a partner’s phallus, men make roughly 170 searches about their own
-
Men’s second most common sex question is how to make their sexual encounters longer. Once again, the insecurities of men do not appear to match the concerns of women. There are roughly the same number of searches asking how to make a boyfriend climax more quickly as climax more slowly.
-
while it’s true that overall interest in personal appearance skews female, it’s not as lopsided as stereotypes would suggest. According to my analysis of Google AdWords, which measures the websites people visit, interest in beauty and fitness is 42% male, weight loss is 33% male, and cosmetic surgery is 39% male
-
you could call it progress that many people today feel they will be judged if they admit they judge other people based on their ethnicity, sexual orientation, or religion. But many Americans still do. You can see this on Google
-
African Americans are the only group that faces a “rude” stereotype
-
In the days following the San Bernardino attack, for every American concerned with “Islamophobia”, another was searching for “kill Muslims”. While hate searches were approximately 20% of all searches about Muslims before the attack, more than half of all search volume about Muslims became hateful in the hours that followed it. And this minute-by-minute search data can tell us how difficult it can be to calm this rage.
-
The “evil” stereotype is applied to Jews, Muslims, and gay people but not black people, Mexicans, Asians, and Christians.
-
Muslims are the only group stereotyped as terrorists.
-
minutes after the media first reported one of the shooters’ Muslim-sounding names, a disturbing number of Californians decided what they wanted to do with Muslims: kill them. The top Google search in California with the word “Muslims” in it at the time was “kill Muslims”
-
Nearly every group is a victim of a “stupid” stereotype; the only two that are not: Jews and Muslims.
-
Parents are two-and-a-half times more likely to ask “Is my son gifted?” than “Is my daughter gifted?”
-
In his speech, the president said: “It is the responsibility of all Americans – of every faith – to reject discrimination.” But searches calling Muslims “terrorists”, “bad”, “violent”, and “evil” doubled during and shortly after the speech.
-
Obama also said: “It is our responsibility to reject religious tests on who we admit into this country.” But negative searches about Syrian refugees, a mostly Muslim group then desperately looking for a safe haven, rose 60%, while searches asking how to help Syrian refugees dropped 35%
-
Obama asked Americans to “not forget that freedom is more powerful than fear”. Yet searches for “kill Muslims” tripled during his speech. In fact, just about every negative search we could think to test regarding Muslims shot up during and after Obama’s speech, and just about every positive search we could think to test declined.
-
new data from the internet, offering digital truth serum, suggested that the speech actually backfired in its main goal. Instead of calming the angry mob, as everybody thought he was doing, the internet data tells us that Obama actually inflamed it.
-
Searches for “nigger jokes” are 17 times more common than searches for “kike jokes”, “gook jokes”, “spic jokes”, “chink jokes”, and “fag jokes” combined. When are these searches most common? Whenever African Americans are in the news.
-
Any theory of racism has to explain a big puzzle in America. On the one hand, the overwhelming majority of black Americans think they suffer from prejudice – and they have ample evidence of discrimination in police stops, job interviews, and jury decisions. On the other hand, very few white Americans will admit to being racist. The dominant explanation among political scientists recently has been that this is due, in large part, to widespread implicit prejudice. White Americans may mean well, this theory goes, but they have a subconscious bias, which influences their treatment of black Americans.
-
There is, though, an alternative explanation for the discrimination that African Americans feel and whites deny: hidden explicit racism. Suppose there is a reasonably widespread conscious racism of which people are very much aware but to which they won’t confess – certainly not in a survey
-
That’s what the search data seems to be saying.
-
this gender bias is not grounded in reality. About 28% of girls are overweight, while 35% of boys are. Even though scales measure more overweight boys than girls, parents see – or worry about – overweight girls much more frequently than overweight boys. Parents are also one-and-a-half times more likely to ask whether their daughter is beautiful than whether their son is handsome.
-
And then there is the phenomenon of Donald Trump’s candidacy. When Nate Silver, the polling guru, looked for the geographic variable that correlated most strongly with support in the 2016 Republican primary for Trump, he found it in the map of racism I had developed.
-
The primary explanation for discrimination against African Americans today is not the fact that the people who agree to participate in lab experiments make subconscious associations between negative words and black people; it is the fact that millions of white Americans continue to do things like search for “nigger jokes”
-
, I was able to use Google searches to find evidence of implicit prejudice against another segment of the population: young girls. And who, might you ask, would be harbouring bias against girls? Their parents.
-
Prior to the Google data, we didn’t have a convincing measure of this virulent animus. Now we do. We are, therefore, in a position to see what it explains. It explains why Obama’s vote totals in 2008 and 2012 were depressed in many regions. It also correlates with the black-white wage gap, as a team of economists recently reported. The areas that I had found make the most racist searches underpay black people.
-
Perhaps young boys are more likely than young girls to use big words or show objective signs of giftedness? Nope. If anything, it’s the opposite. At young ages, girls have consistently been shown to have larger vocabularies and use more complex sentences. In American schools, girls are 9% more likely than boys to be in gifted programmes. Despite all this, parents looking around the dinner table appear to see more gifted boys than girls.
-
What then are parents’ overriding concerns regarding their daughters? Primarily, anything related to appearance.
-
Parents are about twice as likely to ask how to get their daughters to lose weight as they are to ask how to get their sons to do the same
-
Obama’s speech, in other words, was judged a major success. But was it?
-
I did not find a significant relationship between any of these biases and the political or cultural makeup of a state. It would seem this bias against girls is more widespread and deeply ingrained than we’d care to believe.
-
Let’s return to Obama’s speech about Islamophobia. Recall that every time he argued that people should respect Muslims more, the people he was trying to reach became more enraged. Google searches, however, reveal that there was one line that did trigger the type of response Obama might have wanted. He said: “Muslim Americans are our friends and our neighbours, our co-workers, our sports heroes and, yes, they are our men and women in uniform, who are willing to die in defence of our country.”
-
When we lecture angry people, the search data implies that their fury can grow. But subtly provoking people’s curiosity, giving new information, and offering new images of the group that is stoking their rage may turn their thoughts in different, more positive directions.
-
What’s your background?I’d describe myself as a data scientist, but my PhD is in economics. When I was doing my PhD, in 2012, I found this tool called Google Trends that tells you what people are searching, and where, and I became obsessed with it.
-
What would your search records reveal about you?They could definitely tell I’m a hypochondriac because I’m waking up in the middle of the night doing Google searches about my health. There are definitely things about me that you could figure out. When making claims about a topic, it’s better to do it on aggregate, but I think you can figure out a lot, if not everything, about an individual by what they’re searching on Google.
-
All this data I’m talking about is public
-
Does it change your view of human nature? Are we darker and stranger creatures than you realised?Yeah. I think I had a dark view of human nature to begin with, and I think now it’s gotten even darker. I think the degree to which people are self-absorbed is pretty shocking
-
When Trump became president, all my friends said how anxious they were, they couldn’t sleep because they’re so concerned about immigrants and the Muslim ban. But from the data you can see that in liberal parts of the country there wasn’t a rise in anxiety when Trump was elected. When people were waking up at 3am in a cold sweat, their searches were about their job, their health, their relationship – they’re not concerned about the Muslim ban or global warming.
-
Was the Google search data telling you that Trump was going to win?I did see that Trump was going to win. You saw clearly that African American turnout was going to be way down, because in cities with 95% black people there was a collapse in searches for voting information. That was a big reason Hillary Clinton did so much worse than the polls suggested.
