Group items tagged

With the amount of hype around Big Data it's easy to forget that we're just in the first inning. More than three exabytes of new data are created each day, and market research firm IDC estimates that 1,200 exabytes of data will be generated this year alone. The expansion of digital data has been underway for more than a decade and for those who've done a little homework, they understand that Big Data references more than just Google, eBay, or Amazon-sized data sets. The opportunity for a company of any size to gain advantages from Big Data stem from data aggregation, data exhaust, and metadata - the fundamental building blocks to tomorrow's business analytics. Combined, these data forces present an unparalleled opportunity. Yet, despite how broadly Big Data is being discussed, it appears that it is still a very big mystery to many. In fact, outside of the experts who have a strong command of this topic, the misunderstandings around Big Data seem to have reached mythical proportions. Here are the top five myths.

The Department of Education released a draft report about big data and education today. It's called "Enhancing Teaching and Learning through Educational Data Mining and Learning Analytics," a title that's unlikely to win any converts to the notion of a data-curious* view of learning. Part of what's going to get stuck in the craw is that phrase "data mining," I reckon. Despite all the potential and all the buzz about (big) data, data-mining remains something with a fairly negative connotation. Advertisers. Political campaigns. Big government. All sifting through your personal data, trying to uncover the things that nobody knows about, trying to get you to buy or sell or vote. Add to that now the knowledge that every click we make online -- every YouTube view and Facebook like and Google query -- is eminently trackable, it's enough to make all those unsolicited phone calls and junk mail seem quite benign, not to mention old-fashioned.

The worldwide Big Data Week kicks off today with gatherings in the U.K., U.S., Germany, Finland and Australia. As part of their global focus, Big Data Week founder/organizer Stewart Townsend (@stewarttownsend) of DataSift, and Carlos Somohano (@ds_ldn), founder of the Data Science London community and the Data Science Hackathon, have been tracking big data in Europe. This is an area that we're exploring here at Radar and through October's Strata Conference in London, so I asked Townsend and Somohano to share their perspectives on the European data scene. They combined their thoughts in the following Q&A.

"City governments, faced with decreased resources after the Great Recession and rising citizen demand for services with increased urbanization, must be able to make better decisions that are informed by data. To put it another way, in 2012, mayors need to start playing Moneyball in government with evidence-based analysis. From public health to education to energy policy, if governments can shift resources to where they're needed more quickly and accurately, there's substantial positive outcomes for citizens from the application of data for public good. Predictive data analytics - like any data analysis - can only be as effective as the data that they're based upon. Data quality is a long-term concern for any policy maker that wishes to make data-driven decisions, from foreign policy to energy to transportation. If it's bad data, policymakers are going to have a problem, even with superior methodology and algorithms."

Storing huge amounts of data is all well and good but the big advantage of collecting data is using it to help better understand your business, allowing you to become more efficient or to design better products. That has been the message of business intelligence products and data warehouses for years. But the analytics market is changing dramatically, with BI tools working with more data, more integrated analytics platforms, more in-memory databases, a renewed emphasis on real-time data analysis and data visualization, and a recent push toward so-called "predictive analytics."

Le Big Data pourrait devenir la boule de cristal de l'humanité, mais Big Brother et ses apprentis sorciers rôdent dans les coulisses. Le phénomène Big Data comporte sa part de limites, de problèmes d'éthique et de risques qui ne peuvent pas être ignorés.

"There are two sides to the use of Big Data. One is predictive - Twitter has its own sentiment index, analyzing tweets as 140-character barometers. Other companies, like GlobalPoint, aggregate social data and draw algorithmic conclusions. But Big Data has a role beyond digital clairvoyance. It's the role of digital genotyping in the political realm. Simply find the undecided voters and then message accordingly, based on clever connections and peeled-back insights into voter belief systems and purchase behaviors. Find the linkages and exploit them. If a swing voter in Ohio watches 30 Rock and scrubs with Mrs. Meyers Geranium hand soap, you know what sites to find her on and what issues she cares about. Tell them that your candidate supports their views, or perhaps more likely, call out your opponent's demon views on geranium subsidies."

With the increasing digitization of healthcare, the trend of "Big Data" has been gathering steam. According to a new report from digital health consultancy DrBonnie360, there is an estimated 50 petabytes of data in the healthcare realm. That's predicted to grow, by a factor of 50, to 25,000 petabytes by 2020. The report, which I've summarized in this post, does an outstanding job of profiling the leading products utilizing Big Data in healthcare.

"Having worked in academia, government and industry, I've had a unique opportunity to build products in each sector. Much of this product development has been around building data products. Just as methods for general product development have steadily improved, so have the ideas for developing data products. Thanks to large investments in the general area of data science, many major innovations (e.g., Hadoop, Voldemort, Cassandra, HBase, Pig, Hive, etc.) have made data products easier to build. Nonetheless, data products are unique in that they are often extremely difficult, and seemingly intractable for small teams with limited funds. Yet, they get solved every day."

Big data is getting personal. People around the globe are monitoring everything from their health, sleep patterns, sex and even toilet habits with articulate detail, aided by mobile technology. Whether users track behavior actively by entering data or passively via sensors and apps, the quantified self (QS) movement has grown to become a global phenomenon, where impassioned users seek context from their big data identities.

"The healthcare industry is experiencing off-the-charts growth in data generation. Growing numbers of clinical solutions generate more data every day--including electronic medical records, communication systems, and digital image archiving. On top of that,  wearable sensor networks compile information on patients' heart rate, brain activity, sleep patterns, temperature, muscle motion, and numerous other clinically useful data points. This enhanced ability to capture data from everywhere generates massive sets of information. This information is invaluable for healthcare and modern clinical practices--as long as we can manage it properly."

"If the Obama administration's smart disclosure initiatives gather steam, more citizens will be able to do more than think about personal data: they'll be able to access their financial, health, education, or energy data. In the U.S. federal government, the Blue Button initiative, which initially enabled veterans to download personal health data, is now spreading to all federal employees, and it also earned adoption at private institutions like Aetna and Kaiser Permanente. Putting health data to work stands to benefit hundreds of millions of people. The Locker Project, which provides people with the ability to move and store personal data, is another approach to watch."

"What is Big Data? A meme and a marketing term, for sure, but also shorthand for advancing trends in technology that open the door to a new approach to understanding the world and making decisions. There is a lot more data, all the time, growing at 50 percent a year, or more than doubling every two years, estimates IDC, a technology research firm. It's not just more streams of data, but entirely new ones. For example, there are now countless digital sensors worldwide in industrial equipment, automobiles, electrical meters and shipping crates. They can measure and communicate location, movement, vibration, temperature, humidity, even chemical changes in the air."

Over the past several years, I've been writing about how government data is moving into the marketplaces, underpinning ideas, products and services. Open government data and application programming interfaces to distribute it, more commonly known as APIs, increasingly look like fundamental public infrastructure for digital government in the 21st century. What I'm looking for now is more examples of startups and businesses that have been created using open data or that would not be able to continue operations without it. If big data is a strategic resource, it's important to understand how and where organizations are using it for public good, civic utility and economic benefit.

En France, moins d'une entreprise sur dix y voit actuellement un enjeu de compétitivité. Mais dès 2013-2014, le Big Data devrait amorcer une profonde mutation dans l'approche du décisionnel. A la clé, un fort appel à compétences, dont les fournisseurs et prestataires commencent -pour leur part- à se préoccuper.

En France, moins d'une entreprise sur dix y voit actuellement un enjeu de compétitivité. Mais dès 2013-2014, le Big Data devrait amorcer une profonde mutation dans l'approche du décisionnel. A la clé, un fort appel à compétences, dont les fournisseurs et prestataires commencent -pour leur part- à se préoccuper.

The impact of Gartner's top 10 strategic technologies will not center only on the business - their capabilities will also increasingly become integral to future generations of management architecture, according to Gartner, Inc. "We are already seeing the adoption of 'big data' within the IT and operations management [ITOM] industry. In particular, software-as-service [SaaS] management providers now have to collect and synthesize large volumes of data," said Milind Govekar, managing vice president at Gartner. "We also expect more next-generation analytics to come to the forefront to address an increasingly hybrid cloud environment. On the social front, IT service desk social management tools will establish an interactive relationship with end users, enhance end-user productivity, provide a platform to share information and ideas, and market the value of IT to the business." In the fourth quarter of 2011, Gartner identified the 10 technologies and trends that will have the biggest impact for most organizations in 2012. They are: 1. Media tablets and beyond, 2. Mobile-centric applications and interfaces, 3. Contextual and social user experience, 4. The Internet of things, 5. App stores and marketplaces, 6. Next-generation analytics, 7. Big data, 8. In-memory computing, 9. Extreme low-energy servers and 10. Cloud computing.

Deloitte a publié une étude affirmant que l'open data apportera à l'économie du Royaume-Uni non seulement croissance, mais aussi innovation et ingéniosité. Selon ce rapport, l'écosystème open data a des implications fondamentales pour le futur des affaires et la nature des relations client. La tendance vers une plus grande offre et un plus grand usage de l'open data sera irrésistible, obligeant les entreprises à concevoir de nouveaux modèles commerciaux. Elles devront en tirer de meilleurs enseignements, mais aussi développer de nouvelles façons d'échanger avec les parties prenantes et ainsi permettre aux bénéfices économiques et sociaux de l'open data d'émerger.

What data can and cannot do Jonathan Gray argues that aspiring data journalists and civic data hackers should strive to cut back on data-driven hype and to cultivate a more critical literacy towards their subject matter

Un constat : nous sommes entrés dans la révolution Big Data. En 2007, le nombre de données stockées et accessibles était évalué 281 exa-octets. En 2012 ce chiffre sera passé à 1,8 zetta-octets. La quantité de données numériques croît de façon exponentielle, augmentant de 30% chaque année depuis 1999. Les évolutions des usages de la mémoire numérique sont à l'origine de cette explosion : nous enregistrons nos photos, nos vidéos, mais les machines enregistrent également toutes nos transactions en ligne, etc. Cette masse de données représente des enjeux considérables de stockage et de traitement qui sont en passe d'être résolus. Le cloud est une des réponses et le développement de la capacité de mémoire des machines en est un autre, des questions dont nous avions déjà parlé ici. Pour Jean-Daniel Fekete, le principal enjeu de cette masse monstrueuse de données n'est pourtant pas technique, selon lui le véritable enjeu est l'utilisation et l'analyse de ces données.