Why the promise of big data hasn’t delivered yet | TechCrunch

The basic premise of the industry’s offering is this: Hidden in that huge mass of enterprise data are latent patterns. If only you could interpret your data properly, like an explorer deciphering an ancient scroll, you’d be able to unearth these precious business secrets. Specialist analytic software tools are needed to crack the code. The big, diverse, disparate, messy data go into these tools, and “actionable insights” come out. Here is a game you can play at home: Search online for a real-world story of

In the interim, firms have defaulted to leveraging big data in exactly the same way they previously used small data: for reporting and business intelligence. Having invested in purpose-built tools to analyze data at scale, they’ve been rewarded with cool interactive dashboards visualizing it. These are basically auto-generated charts, conspicuously similar to the manually created Excel and PowerPoint reports executives were staring at back in 2005, but far prettier and costlier. It’s easy to see why this approach hasn’t quite delivered on the big data promise.


The beauty of predictive algorithms is that they don’t need to understand the cause and effect behind statistical relationships in order to work incredibly well in practice. For an enterprise to glean the benefits of prediction, it must first give up trying to deduce why things are a certain way, and start trusting the lines of code which tell us that they are.

Source: Why the promise of big data hasn’t delivered yet | TechCrunch

Leave a comment

Your email address will not be published. Required fields are marked *