Concept-Based vs. Keyword-Based Text Analytics; What’s the difference and why does it matter?

A parting gift in the form of a blog post, by our MBA intern, Saman Djabbari.


The average person is thinking one of two things after reading the title of this post. “What in the world is text analytics?” or “You have my attention, what’s the difference?”

Leading industry analyst, Seth Grimes, who in fact has posted on this site before, defines text analytics as “software and transformational processes that uncover business value in ‘unstructured’ text”. Make sense? Not quite? Well, Grimes continues the definition to state that “text analytics applies statistical, linguistic, machine learning and data analysis and visualization techniques to identify and extract salient information and insights. The goal is to inform decision making and support business optimization.” To simplify that, text analytics is here to help you run your business more effectively by ideally saving you time to uncover insights and ideas from your data you might never have been aware.

Now that you get the basics, let’s highlight two of the existing methodologies for text analytics: keyword-based and concept-based. But, before I go into the details of the two, I’m going to provide you with an analogy to help make understanding these methodologies as clear as possible.

Think of some common decisions people have to make in everyday life – let’s think about trying to decide whether to make dinner or order in. By choosing to make dinner you have to purchase ingredients and follow a recipe, which can take a good portion of time. By choosing to order in you typically tell someone over the phone what you want, or you place an order via an app or website. And, are you picking up or do they deliver? For one meal, the cost of each option can often times be comparable, which is why this is a common decision for people, but the true differentiator and pain alleviator is in time and convenience.

Now let’s go into the text analytics methodologies.

In keyword-based text analytics you need to tell the software exactly what to look for. A good example of this is utilizing Boolean logic, which involves typing out a string of words for the software you’re using to detect. Have you ever used Boolean logic? It’s terrible. You have to type in each and every word, permutation of misspelling, any jargon you think might exist, and separating them with conjunctions. There isn’t a greater waste of time than typing in a separate iteration and synonym of a word over and over while separating them with “and’ and “or” repeatedly. Not only must you identify what it is you’re looking for, but also what you’re not looking for, again wasting valuable human hours that can instead be directed toward deriving the insights from the data. Let’s call this the “cooking” method.

Rest assured, there is an easier way. Concept-based text analytics allows you to upload the data to the solution and it will immediately begin to derive insights after a few minutes of processing the data. Sounds much easier, right? Using concept-based text analytics will save you valuable time, not having to tell a system what to search for, and rather let the solution discover new insights for you. Let’s call this the “ordering-in” method.

A perfect example is the work that the Health Media Collaboratory at the University of Illinois Chicago, who is one of our clients, performed using Luminoso’s concept-based text analytics. In order to analyze 140,000 tweets, they simply uploaded the data to the solution all without having to pre-program the solution. The most important themes were delivered to them – “ordering in” their insights if you will.

Stop telling your software what to do and stop wasting money. Go with a concept-based text analytics platform. You’ll save your business thousands of dollars, save time and pain of labor hours, and be more effective in gaining the insight you’ve always been looking for. Don’t get me wrong, I love to cook sometimes…but, you get the point.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s