Group items tagged

How Big Data Became So Big By STEVE LOHR Published: August 11, 2012 This has been the crossover year for Big Data - as a concept, as a term and, yes, as a marketing tool. Big Data has sprung from the confines of technology circles into the mainstream. First, here are a few, well, data points: Big Data was a featured topic this year at the World Economic Forum in Davos, Switzerland, with a report titled "Big Data, Big Impact." In March, the federal government announced $200 million in research programs for Big Data computing.

"Seattle big data startup SpaceCurve announced today that it has set a real-time Big Data performance record for ingesting streaming data and spitting out numbers immediately for queries. SpaceCurve ran a full-blown analytics test on Tweet records that averaged 2,500 bytes in size and included location, user and time information. The rate of operation was done at millions of records per second, hundreds of billions of records per day and petabytes of data per day - a new record, according to SpaceCurve."

Quantity has a quality all its own, as the saying goes. That's especially true of information. The more you've got, the more useful it becomes. Knowing sensitive facts about one person, or a dozen, may be trivially useful. But analyze the same facts about 100 million people, and you can cure diseases, win elections, or earn billions of dollars, because unpredictable insights emerge when you turn computers loose on vast storehouses of information. There's a nickname for the concept: "big data." It's one of the buzzwords of corporate executives, tech-savvy politicians, and worried civil libertarians. If you want to know what they're all talking about, then "Big Data'' is the book for you, a comprehensive and entertaining introduction to a very large topic. By analyzing huge amounts of information, it's possible to discover patterns and relationships that up to now have been invisible to us. In this way, we can find new solutions to tough problems, and opportunities we'd never otherwise have suspected.

Big data might very well be the tech buzz word of the year. And just how hot is it? Consider this:  Data scientist Stephen Purpura - an expert in artificial intelligence, machine learning and predictive analytics who is studying for his PhD in information sciences at Cornell University - has received no fewer than 45 job offers in recent months. And they just aren't any fly-by-night offers, with some rolling in from big-name companies touting salaries of $300,000 or more.

What makes a tweet go viral online? And what does a viral trend actually look like? Those are a couple of the questions that can be answered by a new Microsoft Research project, called Viral Search. The company's researchers are showing the project this week at an internal gathering this week in Redmond. The program crunches large amounts of data from Twitter (and potentially Facebook and other platforms in the future) to analyze and display patterns of distribution on the social network.

You might want to watch the video below before you check in, update your status, or snap and share that photo of you at lunch with your smartphone. The Guardian got hold of this 2010 video demonstration from Raytheon, a big-time contractor that also develops things like missile systems for the Department of Defense, which shows an online tracking tool called Rapid Information Overlay Technology, or RIOT.

The problem isn't just that the study itself is misleading and riddled with errors (which it is). It's that in their efforts to win public favor, the fracking industry increasingly hides behind academia to circulate  misinformation - and the University of Buffalo is the latest cover.