Group items tagged

Big Data is forecast to drive $34 billion of IT spending in 2013 and create 4.4 million IT jobs by 2015, but it is currently still a solution looking for a problem, according to analyst firm Gartner.

While businesses are keen to start mining their data stores for useful insights, and many are already experimenting with technologies like Hadoop, the biggest challenge is working out what question you are trying to answer

Hadoop is designed to efficiently process large volumes of information by connecting many commodity computers together to work in parallel. The theoretical 1000-CPU machine described earlier would cost a very large amount of money, far more than 1,000 single-CPU or 250 quad-core machines. Hadoop will tie these smaller and more reasonably priced machines together into a single cost-effective compute cluster.

You cannot accept problems as handed to you in the business environment. Never allow yourself to be the analyst to whom problems are “thrown over the fence.” Engage with the people whose challenges you’re tackling to make sure you’re solving the right problem. Learn the business’s processes and the data that’s generated and saved. Learn how folks are handling the problem now, and what metrics they use (or ignore) to gauge success.

Solve the correct, yet often misrepresented, problem. This is something no mathematical model will ever say to you. No mathematical model can ever say, “Hey, good job formulating this optimization model, but I think you should take a step back and change your business a little instead.” And that leads me to my next point: Learn how to communicate.

In today’s business environment, it is often unacceptable to be skilled at only one thing. Data scientists are expected to be polyglots who understand math, code, and the plain-speak (or sports analogy-ridden speak . . . ugh) of business. And the only way to get good at speaking to other folks, just like the only way to get good at math, is through practice.

Business Intelligence And Big Data, Q4 2012 survey of 634 BI users and planners, we found that social and mobile data is actually a pretty low priority: While big data technologies like Hadoop can definitely deal with over-hyped new data like mobile and social, the broad demand is simply not there yet. Instead we find leaders recognize that the big deal about big data is the potential for getting more value more quickly from more data, at a lower cost and with greater agility; and it is a whole range of technologies and new techniques that make more possible.

Manipulating Structured Data Using Spark

The amount of data in our world has been exploding, and analyzing large data sets—so-called big data—will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus, according to research by MGI and McKinsey's Business Technology Office.

For example, a retailer using big data to the full could increase its operating margin by more than 60 percent.

important factor of production, alongside labor and capital.