Anyone that works in data analytics and is excited it’s Halloween could appreciate a sign Teradata had displayed near the press room at the PARTNERS event last week in Anaheim, California. The copy proclaimed “The original big data company,” and the image displayed a candy store, with each toffee, lollipop and taffy flavor neatly compiled into its own little jar. Since confectionary stores popped up at the turn of the last century, shop owners had a perfect visual for just how popular each of its types of candy were. It was easy to tell just how valuable each set of candy was to the store, because its sales were visualized in real-time based on how full or empty each jar was.
Getting real-time, visualized analytics is nowhere near as easy today as it was for a candy shop owner 100 years ago, and the problem is unstructured data. In fact, it remains one of the last great impediments of analytics.
Today’s data is more like that giant pillowcase full of a random array of candies that children amass trick or treating. The initial goal was just to collect as much of as possible. But later on, organizations realized it takes a long time to sort through it all and, even after that arduous process, some of it doesn’t even hold value to an organization.
The companies that currently are able to leverage large amounts of data and turn it into insights — even predictions — are using mostly structured or semi-structured data, fueled by AI and machine learning crunching through this information in real-time. As Oliver Ratzesberger said at last week’s “The Sentient Enterprise” breakfast, AI needs a foundation. Companies can’t just hope that whatever they’ve throw into a data lake is going to turn into insights. And AI is proving extremely useful in helping businesses blend their many disparate data silos to gain a better understanding of the complex data ecosystem that now inhabits every major company. They have thousands of those candy jars and need to understand and instantly analyze what that information is doing to their business. But what about all the unstructured data sitting around that’s too time-consuming to sift though?