We had a great week at Strata + Hadoop World in NYC, meeting with over 6,000 data professionals to discuss the future of work and innovation in a world overrun by big data. While there was a lot of great content throughout, we caught wind of a few larger themes that will shape the next few years of analytics.
Here are our top 4 takeaways from the event that we’ll keep in mind now that we’re back to work in Palo Alto
1. Companies need a better way to put big (and small) data in the hands of business users.
We were pleasantly surprised to see folks from so many different walks of life at the conference. It’s clear that big data has become mainstream - everyone wants a piece of the pie. Unfortunately, access to insights isn’t so simple. The issue here is two-fold. First, there are too few data analysts to meet the reporting demands of the rest of the business. As a result, companies are struggling to translate all of their data into meaningful and timely insights. Second, tools that aim to fix this middleman issue - i.e., offer an “easy” or “intuitive” user experience for all - actually remain inaccessible for non-technical users. Not to beat a dead horse here, but these false claims are really why business intelligence adoption rates are fixed at 22%. Big data is right in front of us all, but accessibility needs to improve.
2. Data governance is more important than ever.
When you’re dealing with corporate or sensitive data, you have to make sure the right people - and no other people - have the right access to it. But then you also need a single source of truth. This becomes trickier as we have moved to a world where combining multiple data sources is the new norm, and so balancing all of these different needs can be a real challenge. When you analyze your data, you need one answer that you can count on, not a few different answers or estimates. Really, both points can’t be overemphasized - Mike Olson even mentioned it in his keynote to kick off the conference. Cloudera’s RecordService Project is a new security layer for Hadoop designed to ensure that anyone can get a consistent, secure, appropriate view of that data. Security needs to be a top priority for both vendors and buyers.
3. Big data prep has always been a challenge. Vendors are finally stepping up.
The quality of data insights rests on the quality of the data. We saw lots of “data janitor” companies at the show. Within this category, we noticed the rise of new tools offering business users the chance to do their own prep. Linking back to our first theme, vendors are finally understanding the need to put more power in the hands of business users. This is crucial because most big data isn’t even useful. It’s dirty and must be thoroughly cleaned in order to separate the signal from the noise. Giving each individual - as opposed to only analysts - the resources to identify and fix anomalies in the data will make every company more efficient and successful.
4. Buyers are expecting instant insights, and streaming is becoming mainstream.
As Forbes predicted over a year ago, the new rule for the future is, “anything that can be connected, will be connected.” This Internet of Things craze has a lot of us excited about opening up new access to data at each moment. We heard tons of chatter at Strata around IoT possibilities and streaming data from multiple platforms - buyers are excited about the promise of bringing real-time analytics to their employees, and are asking vendors to get on board.
The future belongs to those who know how to get the most out of their data. While the big data hype has always hinted a more innovative and iterative future for enterprise technology, a recent and growing focus on the end user is a move in the right direction. Companies are looking to put easy data prep, seamless data access and instant insights in the hands of all users without compromising security. Sounds about right to us.
