SOCAP International

The Dirty Little Secret of Text Analysis

Here are the tips you need for maintaining the viability and effectiveness of a text analysis application.

Click here to access the full digital issue of CRM Magazine Winter 2015

Potts “That’s the dirty little secret!” was the response to the last question posed during a presentation and discussion about the challenges and opportunities of monitoring, managing and responding to social media at a recent international conference of consumer affairs professionals. After an excellent review of the complex planning, executive involvement and buy-in, and positive results of automating consumer feedback analysis, the attendees were offered an opportunity to ask questions. The final question was about the burden necessary to maintain a free-form text analysis environment. The moderator explained that free-form text analysis requires a significant and growing burden of upkeep to maintain the relevance of the results from the analysis.

High Maintenance

Text analysis is a complex process. Unlike most software solutions, text analytics requires frequent upkeep that may not be evident when vendors demonstrate their wares, and it’s certainly not something most vendors talk about. An initial demonstration of a text analytics solution using a prospect’s data can be an eye opening experience. The dirty little secret is that, as an organization starts seeing the potential, it demands greater specificity from the solution, and that requires upkeep—often lots of upkeep.

Let’s take a step back and look at the steps required to make free-form text useful.

Unless a company’s business, products, markets, customers, surveys, etc. do not change over time, text analysis requires a significant amount of time and attention to derive value from free-form text. Unlike other software categories, text analysis requires ongoing upkeep if it’s to deliver accurate, consistent and relevant results.

When companies turn to text analysis software, they’re often trying to:

  • gain insights and greater value from their data
  • improve accuracy and consistency in their results
  • relieve the burden of reading and scoring or categorizing user-generated content and free-form text from surveys, social media, call center and chat interactions
  • improve throughput by automating the process.

Often the first time data is processed with a text analytics software, the results are impressive, and the assumption is that, when the software is put in place, it will continue to provide valuable insights. In reality, that will only be possible if the processes that support the software are maintained.
In many cases, the unanticipated burden or “dirty little secret” of text analysis is the upkeep involved in refreshing the models or rules some software requires, maintaining and updating the terms and topics, and tweaking the concepts and

An organization has a number of options for the upkeep involved in maintaining the viability and effectiveness of a text analysis application.

1. Recruit and hire staff that has the expertise and experience with text analytics and a particular software.

This is a time-consuming endeavor that often becomes a frustrating solution. Unlike other recruiting and hiring situations, management may not know the skills and experience a new hire must have to successfully maintain a text analysis application. Additionally, this role may not require a full-time employee and adds to the overall cost of the project.

2. Develop the expertise internally and grow the experience in-house.

This is a long-term strategy. Because the text analysis upkeep role is often part time, employee retention becomes an issue. As the value of text analysis becomes evident and the internal demands for more projects increase, costs escalate, and turnover can be problematic. Additionally, companies must find a vendor that has the resources and is willing to train and coach a client’s staff on its particular software and on the skills needed to understand the subtleties involved in text analytics.

3. Hire an agency to maintain the text analysis application.

Selecting an agency to supply and maintain a text analysis application as part of its services is a challenge. Agencies often specialize in marketing services (including the design and execution of direct and digital marketing campaigns, advertising, branding and survey design). And while they offer a wide range of services, their expertise in text analytics may be thin and offered as a way to present a one-stop solution. Text analytics requires a deep understanding and familiarity with the technology that is beyond most agencies’ level of expertise.

4. Contract with the software vendor to supply services.

Contracting with a software vendor to supply upkeep services is often the default solution that becomes necessary after an organization buys an analytics software solution. For many organizations, however, the desire or need to improve text analytics motivates them to select a vendor’s software without a clear understanding as to the level of upkeep required to maintain the software’s effectiveness. This is especially true when text analytics software is a rules-based technology, as most are. They require rules to be built, maintained and updated for the software to extract relevant information from free-form text.

As a company’s products, packaging, markets and messaging change, the rules need to be updated to capture that new information. The cost to have the vendor do the rule building that is necessary to maintain the relevance of the results from the software can be high, and yet may not entirely relieve their customer of the burden, especially if the vendor doesn’t typically offer those services or have a broad range of experiences with many types of industries to offer a diverse range of best practices.

5. Select a vendor based upon a Solution as a Service model.

Potts_4This model bundles the expertise and broad experience necessary to help a client quickly achieve their goals by providing whatever level of service the client requires. That can include the vendor offering their software, training and services on a project-by-project basis to help the client prove the value of text analysis to their executive management team.

Whether it is a full-service model in which a vendor delivers analytic results through a number of different user interfaces (e.g., interactive PDFs, dashboards, scorecards and custom reports) or it works with the client to develop and enhance their own staffs’ skills, the vendor maintains the analytic environment and does the necessary upkeep to make sure the client obtains the greatest return on investment in text analytics. 

At the end of the day, a client wants value from their investment in text analysis. An ethical software vendor should inform the buyer about the benefits of and challenges inherent in maintaining a text analysis software.

Text analysis offers significant value if done right. Buyers need to factor in the costs of upkeep as they consider various solutions and vendors. There is no free lunch. As in any relationship, it behooves the vendor to be candid about the barriers to a successful long-term text analysis implementation, and it requires the client to candidly assess how much of the upkeep burden accompanying any successful text analysis implementation they are willing and able to take on.


Potts joined PolyVista in 2003 and was appointed vice president responsible for sales shortly thereafter. Currently, he spends much of his time working with companies to help them maximize the benefits they derive from data analysis. He began his involvement in the business intelligence market as an early employee of the company that helped coin the term OLAP (on-line analytical processing) in 1993.