This article originally appeared on business.com.
When Walmart announced a new search technology in 2012, Neil Ashe, president and CEO of Walmart Global E-commerce, said, “Search is a crown jewel for any E-commerce company to own.”
This statement is even more relevant in 2016 as every aspect of online retailing continues to grow. The number of retailers, shoppers and products are up, as is the amount of data that an online retailer’s search engine must wade through to present shoppers with products and recommendations.
Yet even as retailers are engaged in a continuous battle to optimize the shopping experience, they have a dirty little secret in the back office. Their ability to get insights from the growing amount of data being collected is limited by the limitations and complexity of the retail analytics tools that merchants have access to.
Traditional Methods Analytics
While merchandising teams run standard database queries to produce their weekly and monthly BI reports from traditional Business Intelligence solutions such as BusinessObjects and Tableau, the amount of data now being stored has forced many of them to employ the 80/20 rule, focusing their analytical efforts only on the top and bottom 20 percent of their products. It has simply become too expensive and too cumbersome to adequately analyze the full product portfolio.
But this leaves a significant amount of insight, product optimization, and ultimately revenue they cannot generate.
For example, what happens when the CEO suddenly wants to understand the underlying reason for the poor performance of exercise equipment in a certain region last quarter in order to make a future investment decisions? Or how does the vice president of merchandising understand why particular regions of the country experienced a sudden dip in sales of a particular brand of headphones relative to the previous quarter?
Ad hoc queries are required to answer these questions, which means most merchandising teams are forced to ask a data analyst to construct and submit a query, that can take hours or even days to bring back results, while the executive management team is paralyzed and only grows more frustrated. By the time they get their reports a week later, they have already had to make their decisions blindly. This situation is particularly hard on the merchants who make the buying decisions.
I know one senior merchant who looks after all of men’s apparel at a major retailer, and she would start several days each week by running a simple query first thing in the morning and then go out for breakfast while it ran. Leaving the office was far better to waiting for an angry manager to complain about the delay in getting them answers.
In addition to these reporting delays, frustration with existing clunky analytics tools often forces merchants to copy subsets of data from the data warehouse onto their desktops. They may wait for a long query to run and then extract the results into Excel, or they may ask the IT team to create a SQL query that dumps out a .CSV file for them to play with.
But this supposed short-term gain only causes more long-term pain as the data they extract, ungoverned by IT, quickly becomes dated, and the organization soon ends up with multiple versions of the same datasets, causing confusion. This “Shadow IT” inevitably leads to unreliable insights, and the existence of data on a variety of devices precludes IT from having the necessary data governance over who has access to what data.
The other emerging challenge for merchants is that traditional business analytics tools make it extremely difficult for organizations to combine their sales data with the large volumes of unstructured data they collect, such as social media streams. While capturing big data in real-time is now feasible, analyzing it easily along with traditional data sources is anything but.
Next-Generation Search-Driven Analytics
Fortunately, in the same way that next-generation search technology is revolutionizing the online shopping experience, it has the potential to allow merchants to get answers to all their questions about all their data, structured and unstructured, in seconds. While making it easier and faster to access data, these solutions can also provide IT with the ability to govern who can access and analyze individual datasets.
Next-generation search-driven analytics technology is made possible in part by hardware improvements that enable in-memory databases to hold data in large amounts of RAM for dramatically faster query processing, seconds instead of hours.
Such solutions can also typically incorporate all forms of data, structured and unstructured, and from on-premises, desktop, and the cloud. The speed also enables retailers to automatically update standard queries and charts each time the application is opened. Over are the days when users had to click back through set-up trees or rerun queries and wait additional hours for the updated information.
In addition to these general capabilities, the benefits for those who actually work on the data can be enormous. With these new solutions, users are able to ask follow-up queries immediately after getting an answer to the prior question, which preserves continuity of thought.
The speed of processing also enables these search solutions to make calculations on the fly, so the query about the performance of exercise equipment by region by quarter, for example, doesn’t just provide a subset of data to be further analyzed. Instead, it can help optimize inventory levels by providing the actual percent of growth or decline by each region.
The use of search-driven analytics technologies is gaining steam, and, according to Gartner, “Data discovery will continue to displace IT authored static reporting as the dominant BI and analytics user interaction paradigm.”
Thanks to this new paradigm, the merchant who used to go out for breakfast while waiting for results now gets her answers in seconds, which has given her time to analyze more products. Applying the 80/20 rule is no longer necessary because she is able to analyze her entire product line and make optimal buying decisions that have dramatically cut costs and increased revenue.
Also, thanks to the Google-like ease of search-driven analytics, non-technical salespeople and even executives are able to construct their own queries, without any analyst assistance or training. They get the answers they need immediately and can make informed business decisions and move on to other tasks.
It’s important to note that search-driven analytics technology isn’t just about analyzing data faster to facilitate more timely decision making. The real-time analysis of both structured and unstructured data is also enabling retailers to understand the relationships between sales figures and the vast array of social media discussions that take place every minute of every day, often on a global scale.
For example, a retailer can now directly analyze the relationship between social media sentiment about a product in various regions around the world and the sales and inventory data living in the data warehouse.
Retailers collect enormous amounts of structured and unstructured data so they need solutions like this that are both easy for front-line merchants to use and massively scalable. Search-driven analytics offers retailers the opportunity to shine a new light on back office operations, providing immediate insight into all the data they now collect.
Fast and easy to use, the technology means that retailers can realize many retail analytics benefits, such as making better and more timely decisions about their entire product lines, eliminating waste, increasing revenue, and better meeting the evolving needs of their customers.
