Bridging the Automation Gap to Meet Today’s Insights Demands

Market research is speeding up. Demands on researchers are rising. Pressure is increasing for granular, quality insights. We know these things to be true and yet many still struggle with implementing the technology needed to meet this new reality head-on.

Too often, there is an internal disconnect at many companies. This is one of the first gaps, if not chasms, that crop up along the way. Business leadership may have a long-term vision for efficiency and future-proofing that is not aligned with the operations and IT teams’ interpretation of how best to fulfill this vision. For the latter two teams, the most pressing, immediate issues tend to be prioritized and, consequently, automation solutions tend to be designed and architected to meet relatively short-term goals. 

This sets in motion a journey that may ultimately be destined for catastrophic failure and major expense. And this journey could quietly cost the company its long-term ambition and ability to compete, as innovation dies under a mushroom of technical debt.

KEVIN BHAGAT, UNSPLASH

In our industry, we see this happening most frequently in the large, broken, and largely unresolved piece in the middle of the market research process, having to do with supplier and fieldwork management. While many are using the power of automation to tackle challenges in steps such as survey programming and design all the way through to data analysis and reporting, the data collection piece often becomes the bottleneck. This is the second gap, and it’s one that promises to become even wider as time goes on.

When companies try to solve the data collection sticking point internally, the focus is taken away from the company’s primary vision and goals. What many fail to realize initially is the massive complexity of this stage. In order to find an acceptably-sized pool of respondents to achieve quotas, feasibility, and data quality, most need to tap into multiple suppliers. There is no standardization among these suppliers, which makes combining them a full-time, specialist task. And that’s just a tiny portion of the critical components that must be considered. Companies must also manage financial and process impacts, effects on stakeholders, efficiencies of the entire supply chain, human resources strain, and much, much more. It’s easy to see how maintaining this piece of the process quickly takes over and introduces gargantuan disruptions to daily workflow. Long-term implications abound.

The truth is, however, that solutions are out there. Software, APIs, and algorithms now present a better way. Technology allows companies to tap into the power of existing solutions and partners that can handle all the moving pieces for them, in an adaptable, seamless way. For example, solutions already exist that automatically standardize all of the considerations from a vast number of sample suppliers, panels, and third-party vendors. This takes the impossible burden of standardizing respondent profiling and other supplier-centric complexities off internal shoulders. This alone can allow companies to renew their focus on being innovative, agile, and fast-to-market, which is becoming more and more important in today’s ecosystem. 

MAROS_BAUER, ISTOCK x YOD67, ISTOCK

Automation truly is a business-critical issue in the market research and consumer insights industries, especially at a time when pressure for agility and efficiency is increasing all the time. The decisions insights companies make today regarding automation will impact future success or failure. We’ve outlined these complexities in a recent white paper, Transforming data collection without paralysing your insights business.

Today, success means employing genuinely scalable and future-proof solutions that are equipped to address the unique challenges faced by market researchers, preparing them for the next phase of change. On this front, huge wins can be realized by bridging that data collection step with smart automation – but only when it is done in a way that doesn’t take the proverbial eye off the prize: a company’s long-term vision and goals.