What have you searched for today?
Did you research how to do your job better? Find a new coffee machine? Locate a great Thai restaurant near work? Look up an ex-coworker?
Pretty similar experience. But behind the scenes, each of these searches works very differently.
The Relational Search Problem
All search is not created equal.
Google uses document search, because web pages are basically souped-up text documents to a computer. PageRank takes into account the links between these documents, so the most relevant ones bubble up to the top. Over the years, they’ve added some not-so-secret sauce, so that search optimization experts will never lack for work.
Amazon uses “faceted” or “object” search. They know that you’re searching for a product. So for them, the searchable universe is a known set of objects and their properties. On LinkedIn, the objects are people, companies, and jobs.
There’s also graph search, which uses the connections between entities (or friends, if you’re Facebook). Facebook graph search lets you specify how broad your search should go within your friend network.
Introducing relational search.
We’re exploring a totally new kind of search called “relational search." Relational search may sound like a warped version of a 1970s candy bar collision.
But it really just means using search to analyze your company data from financial, marketing, sales, supply chain, and other business data sources.
Search is hard. And it gets harder when you try to apply it to enterprise data.
It's a different kind of search problem because your company's data is nothing like the Web documents you search with Google, or the network of your friends on Facebook. The most valuable data in your company is probably stored in relational databases, cloud applications, Hadoop clusters, and even spreadsheets.
Here’s what makes relational search hard(er) than some of the existing types of search.
Why is relational search a hard(er) problem?
Your company’s data is complicated.
When you search on LinkedIn, you’re probably looking for a person or a company. These are standard “objects” that LinkedIn search engine can understand.
But a company’s data is much more complicated. It spans multiple databases, tables, columns, rows, and keys, with a spaghetti of relationships between them. A relational search engine needs to crawl your data sources, understand these relationships, and allow you to search and analyze this data. All this must be done without losing the experience people have come to expect from a search bar.
The results have to be 100% accurate.
Sometimes even Google gets it wrong. If you search for “cities that are not in California”, you’ll get these results. You don’t even have to click that link, because you already know that you’ll get lists of California cities.
But the implications of a bad Google search are trivial. You just reword your search and try again.
Not so with relational search. You need an accurate answer every time, or you risk bad business decisions. What’s worse than guessing? Being convinced by bad data.
You’re not allowed to see everything.
Parental filters aside, most of what you search online doesn’t have to be secure. Not so for the data in your enterprise.
Security has to take into account your role, your department, and sometimes your geographical location. It’s easy to hide a column, like salaries, but what if you want to only show sales managers information on their region? That requires row level security.
It needs to be fast.
Search engines need to respond to users instantly (often measured in milliseconds). Here are the things that a relational search engine needs to do within a very short time:
Read each of your keystrokes
Find what you are typing in a huge morass of data, including the relationships between the data elements
Apply security rules
Make relevant suggestions
Translate your search into queries that can be applied to your data
Bring accurate results back to you
And that’s just the search experience. You might be willing to wait two weeks for a report, but you expect search suggestions to be instant.
There’s a lot of stuff to search.
Searching enterprise data isn’t like searching the file system on your laptop. We’re talking about terabytes of data, spread across multiple databases that included hundreds of tables.
In Web search, the data volume is huge, but a single search only touches a tiny fraction of that data. If you type "kittywampus," only the Web pages that include that term are included in the search (that’s 54,100 pages, if you’re counting).
In relational search, every data element has to be considered for matches with each search term. And often, large amounts of data must be aggregated, as in the search “revenue last 5 years.”
But we like hard problems.
Why did we leave companies like Google, Amazon, LinkedIn, Facebook, Oracle, IBM and VMWare to gather around the ThoughtSpot fire? We were lured by the idea of building something we hoped would make a difference.
And we like working on hard problems.
“We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard.”
-John F. Kennedy, Address at Rice University, September 12 1962
First of all, this is a misquote. He really said “...because they are haaaaaad.” But whatever.
Second, what is Kennedy talking about when he says, “do the other things”? In the preceding sentence, he refers to the following tasks as "hard things":
Climbing the highest mountain
Flying the Atlantic for the first time
Rice playing Texas (Kennedy was speaking at Rice University, which had just donated the land for the Johnson Space Center and established a space studies program.)
The world of high tech looks a little different than it did in 1962. But some things haven’t changed at all. Solving hard problems still stretches our brains and bonds us together.
So, yeah. We like hard problems. Our brains like hard problems. And we want to solve them with other people who like hard problems.
We believe that relational search can make a difference. Come join us as team members, partners, customers or just well wishers. This is a massive undertaking and we need all the help we can find.
Not clear why relational search is the way to go? See our post on why BI sucks.
<br>
