Survey Respondents: The Polar Ice Caps of Market Research

 

By Lisa Wilding-Brown

In partnership with my esteemed colleagues, I had the fortunate opportunity of developing one of the industry’s first online panels, The Harris Poll Online. Since then, I have built hundreds of panels. Mothers, gamers, business professionals, mortgage brokers, teens, high income earners, GLBTers, physicians and the list goes on. I’ve recruited every audience you can imagine. With all this in mind, I have some ideas about panels and the research ecosystem that may be hard for you to hear. Sit down, grab an ice-cold beverage and buckle up. I’m going to serve you the hard truth. I’m not doing this to win or lose friends. My goal is laser-focused. I want to see this industry survive. I want to see research budgets increase and marketers around the world to rely on our space for trusted, reliable insights for the long-term.

Let’s kick-off the conversation with a story that I think will crystalize the discussion perfectly.

Several years ago, a company that will remain unnamed, proclaimed a bold and controversial marketing message, “Panel is dead.”  At the time, my COO and other senior members of the team were outraged. “How could ‘company X’ say such a thing?!” As a result, the bold and brazen voice was silenced and our leadership relegated this firm to the penalty box.

In reality, panel wasn’t dead in the real sense. After all, the number of panels in the industry were increasing at an exponential rate. When I started in the early 2000s, there were only a couple of sample companies. In 2016, there are over 63 sample companies listed in Greenbook and I believe this number only scratches the surface. Indeed, panel is not dead, but… panel is very sick. Deathbed sick. Now hold on… let me explain before you demote me to the penalty box.

It is not unusual to hear industry bloggers and conference speakers discuss “declining response rates” or “respondent burden,” but we have to ask ourselves the question, “Do we really care?”  The short answer? ”No.” Most of the industry thinks of respondents as a commodity – an infinite resource that will always be available for our use. Plainly put: respondents are the polar ice caps of Market Research. We know they are important, but we aren’t doing much to reverse the damage of our daily abuse. Of course, we talk a LOT about the impact of long surveys, shrinking incentive budgets, hostage-taking routers, and price compression. However, at the end of the day, nothing is changing.

As a sampling professional, I feel constant pressure to reduce my pricing to compete with what I call “faux sampling companies.” These are companies that pretend to be sampling experts. Unfortunately, under the covers they are nothing more than online traffic pushers. Rain collectors with no real experience or heritage in Market Research. The market has been flooded with these types of companies over the last five years. As a sample buyer, clients must feel overwhelmed and confused by their options. These faux samplers can herd large flocks of online traffic and they know just enough MR vernacular to be dangerous. While their ability to access online traffic is impressive, we all know that sampling is much more than “driving traffic.”  Let me be clear, sampling experts can develop and replicate complex sample frames which yield representative results; traffic pushers do not.

I have to look at the bigger picture and focus on longevity and quality. “Don’t get caught up in the short-term competition,” I frequently tell myself. I have to admit, I wasn’t so strong in the past. My former CFO and board would come to me and demand savings. I would do my best to explain the long-term impacts of short-term thinking, but this fell on deaf ears. Cutting incentives by 30% doesn’t make sense. Adding routing cycles in order to increase conversion kills long-term retention. This failed routing approach produces bias and burns out respondents; and yet we see both real (and faux) sampling companies doing this every day. Enough is enough. The ice caps are melting, the water is rising and most sampling companies are drowning… or merging. Same thing.

So, how do we fix this?  How do we increase response rates?  How do we increase the pool of people who will even tolerate our surveys?  In order to improve our industry’s outlook, we need to make some significant changes across the entire research ecosystem. Here are some ideas to get you thinking…

1. Survey design is a skill. It requires training and discipline. Just because you can program a sixty-minute survey, doesn’t mean you should. Like all things in life, we must negotiate. We must serve our clients as consultants; providing advice and evidence as to why a sixty-minute survey is an irresponsible idea.

2. Excluding mobile respondents from your sample is another damaging idea. Stop doing it. If your survey isn’t mobile-friendly, you are doing it all wrong. Wake up from your desktop coma. Mobile is happening now.

3. We must remember that “sample” are actually people. Let me ask you this, would you take a sixty-minute survey for $1 or the equivalent in some virtual currency? Probably not. Would you enter an endless router that holds you captive for twenty minutes only to disqualify? Probably not. We have to ask ourselves these common sense questions every day. Otherwise, we will continue to operate in our alternate universe, huddled in the fetal position waiting for our life raft!

4. Most panels are truly broken. They are the shards of a panel manager’s broken dreams! Okay, I know what you are thinking…maybe that is touch dramatic, but it can be a depressing job at times.

Some real-life scenarios from my past life:

1. It doesn’t make sense to change your incentive model so that it is humanly impossible for a member to redeem a reward. Yes, you will save on your monthly redemption costs, but no one will stay in your panel. As a result, your recruitment costs and outside sample spend will go through the roof.  You will spend more and save less.

1. It doesn’t make sense to build a panel model where members only have a positive experience 10-20% of the time! Let’s distill that down for a minute: most panels only enjoy a 10-20% conversion rate (successfully completing a survey). This means that panelists are having a disappointing experience 80-90% of the time!

Who in their right mind would stick with something that produces such a negative outcome? I can tell you who: a VERY tolerant and eccentric individual who doesn’t mirror the general population whatsoever. As a sampling expert, I can tell you this is very problematic.

While another shard of an ancient glacier breaks off into the sea; you’ve likely finished your ice-cold beverage by now. I’m sorry if this blog drained you of your life force. The truth hurts, but being in a potentially obsolete industry hurts more. We have to make changes to reverse the damage and my team is setting out to do just that. We are redefining the user engagement model.  Join me in the #MRXRevolution!