Group items tagged

The beauty of such Big Data applications is that they can process Web-based text, digital images, and online video. They can also glean intelligence from the exploding social media sphere, whether it consists of blogs, chat forums, Twitter trends, or Facebook commentary. Traditional market research generally involves unnatural acts, such as surveys, mall-intercept interviews, and focus groups. Big Data examines what people say about what they have done or will do. That's in addition to tracking what people are actually doing about everything from crime to weather to shopping to brands. It is only Big Data's capacity for dealing with vast quantities of real-time unstructured data that makes this possible.

the number of Google queries about housing and real estate from one quarter to the next turns out to predict more accurately what's going to happen in the housing market than any team of expert real estate forecasters. Similarly, Google search queries on flu symptoms and treatments reveal weeks in advance what flu-related volumes hospital emergency departments can expect.

Much of the data organizations are crunching is human-generated. But machine sensors — what GE people like CMO Beth Comstock called "machine whispering" when I talked with her this past summer — are creating a second tsunami of data. Digital sensors on industrial hardware like aircraft engines, electric turbines, automobiles, consumer packaged goods, and shipping crates can communicate "location, movement, vibration, temperature, humidity, and even chemical changes in the air."

Two years ago, the two greatest Jeopardy champions of all time got obliterated by a computer called Watson. It was a great victory for artificial intelligence--the system racked up more than three times the earnings of its next meat-brained competitor. For IBM’s Watson, the successor to Deep Blue, which famously defeated chess champion Gary Kasparov, becoming a Jeopardy champion was a modest proof of concept. The big challenge for Watson, and the goal for IBM, is to adapt the core question-answering technology to more significant domains, like health care. WatsonPaths, IBM’s medical-domain offshoot announced last month, is able to derive medical diagnoses from a description of symptoms. From this chain of evidence, it’s able to present an interactive visualization to doctors, who can interrogate the data, further question the evidence, and better understand the situation. It’s an essential feedback loop used by diagnosticians to help decide which information is extraneous and which is essential, thus making it possible to home in on a most-likely diagnosis. WatsonPaths scours millions of unstructured texts, like medical textbooks, dictionaries, and clinical guidelines, to develop a set of ranked hypotheses. The doctors’ feedback is added back into the brute-force information retrieval capabilities to help further train the system.

For Watson, ingesting all 2.5 million unstructured documents is the easy part. For this, it would extract references to real-world entities, like corporations and people, and start looking for relationships between them, essentially building up context around each entity. This could be connected out to open-entity databases like Freebase, to provide even more context. A journalist might orient the system’s “attention” by indicating which politicians or tax-dodging tycoons might be of most interest. Other texts, like relevant legal codes in the target jurisdiction or news reports mentioning the entities of interest, could also be ingested and parsed. Watson would then draw on its domain-adapted logic to generate evidence, like “IF corporation A is associated with offshore tax-free account B, AND the owner of corporation A is married to an executive of corporation C, THEN add a tiny bit of inference of tax evasion by corporation C.” There would be many of these types of rules, perhaps hundreds, and probably written by the journalists themselves to help the system identify meaningful and newsworthy relationships. Other rules might be garnered from common sense reasoning databases, like MIT’s ConceptNet. At the end of the day (or probably just a few seconds later), Watson would spit out 100 leads for reporters to follow. The first step would be to peer behind those leads to see the relevant evidence, rate its accuracy, and further train the algorithm. Sure, those follow-ups might still take months, but it wouldn’t be hard to beat the 15 months the ICIJ took in its investigation.

When I ask participants why they’ve chosen to receive emails from a particular source, read a specific blogger, or follow a certain Twitter user, they give me a variation on the same answer: “Because I like their unique point of view.” Readers will only listen to you if you’re giving them something they can’t find anywhere else.

My numbers-based research has confirmed the importance of uniqueness and novelty. The data shows that novelty is contagious; ordinariness is not.

Tweets with uncommon words get Retweeted more often than the usual things we see every day. Having a unique way of expressing yourself will earn you more Retweets.

According to GWI, mobile access to social media sites actually overtook traditional PC access in Q4 of 2013, as 66 percent of users accessed their social networks by mobile compared to 64 percent by computer. However, microblogging sites — which include Twitter and Tumblr — are apparently best reserved for the tablet, dominating over both traditional computers and mobile for usage.

No matter what the device, Facebook remains top dog across the board overall – account ownership, active usage and visit frequency, across all regions — although it has seen minor decline as other social networks gain mindshare. The key winner in this year’s new class of social networks is Instagram: A nearly 25% rise in active users betwen Q2 and Q4 of 2013 bring the estimated total of active users on the website to more than 90 million. It’s also popular for the kids, too, as teens represent the dominant demographic on the site, with a 39 percent share of active users. According to GWI, the only other social networks that can boast teens as their dominant users are Youtube and Tumblr.

GWI’s data only indicates that Facebook’s teens shrank two percentage points, leaving a rough user estimate of 34.19 million

Pinterest has beaten out Yahoo organic traffic, making Pinterest the fourth largest traffic driver worldwide

Google, Yahoo, and Bing organic traffic decreased by 15.63% on average since January, which the firm speculates may indicate more people are discovering content through social sites like Pinterest.

it could also be because Shareaholic’s data, which comes from a network of 200,000 publishers using its social sharing and content analysis tools, is more likely to reflect an engaged community where people are comfortable with using social networking sites to perform searches. In other words, it’s not a big picture study here – just a slice.

Forrester Research defines digital intelligence this way: The capture, management and analysis of data to provide a holistic view of the digital customer experience that drives the measurement, optimization and execution of marketing tactics and business strategies."

the specialist era is waning. The future may belong to the generalist.

there appears to be reasonable and robust data suggesting that generalists are better at navigating uncertainty.

Professor Phillip Tetlock conducted a 20+ year study of 284 professional forecasters. He asked them to predict the probability of various occurrences both within and outside of their areas of expertise. Analysis of the 80,000+ forecasts found that experts are less accurate predictors than non-experts in their area of expertise.

100 characters is the engagement sweet spot for a tweet. 

a spike in retweets among those in the 71-100 character range—so-called “medium” length tweets. These medium tweets have enough characters for the original poster to say something of value and for the person retweeting to add commentary as well.

the ultra-short 40-character posts received 86 percent higher engagement than others.