GRIT Says Panel Woes Are Jeopardizing MR’s Future.

By Michael Smith

A running theme through the 82 pages of the most recent Greenbook Research Industry Trends Report (GRIT) is that the quality of survey sample has eroded to the point of crisis for market research.

GRIT sounds a repeated alarm over what its authors call “a known problem…with no solution in sight.” But there is a solution — all-mobile panels — which I’ll explore in a bit.

First, some facts and quotes from the report that lay out the dimensions of the panel crisis:

• 38% of GRIT’s more than 2,000 industry respondents expect sample quality to get worse over the coming three years; fewer than 28% believe it will improve – and among clients of market research providers, optimism sank to 23%.

• “Clients and suppliers agree that sample quality is getting worse, and there is little alignment on what to do about it. This is a perennial topic; when will the industry do something about it?”

• “The smartphone revolution and declining participation are indeed problems that need to be addressed. Few disagree with this belief, but there is far less consensus around the extent of the problem, its implications and the range of solutions.”

• “The difficulty of accessing truly representative sample sources….could be viewed as the single largest area of concern for the industry….We are running out of online panelists…”

• “There are few legitimate excuses one can muster for not confronting the sample problems that plague the industry. There’s no doubt that the solutions are hard, but…far too many people…are dragging their feet.”

• “The real existential threat to our industry is…the future of research participation. The real question therefore is when will people catch on? When will responses to these questions drive change?”

• “We believe that the death spiral is accelerating for those researchers who fail to act. The poor experiences they create are starting to contrast markedly against the unique and engaging experiences by new entrants as well as the small number of innovators who’ve been unafraid to embrace change.”

The last sentence points the only way forward. Innovate. Embrace change.

A Formula for Successful Mobile Research

My argument is that state-of-the-art mobile research is the innovation our industry needs to embrace. But before that can happen, we have to overcome a common misconception about what mobile research really is, and what it can accomplish.

The misconception is borne out by one of GRIT’s most telling findings: 74% of respondents think they’re already doing mobile research, more than any other “emerging method.” An additional 17% are considering trying mobile for the first time.

MFour has long struggled to make the industry realize that not all mobile research is created equal. There’s good mobile and bad mobile, mobile that’s artless and mobile that’s state-of-the-art. There’s pure mobile that’s solely geared to smartphones, and diluted mobile that ties smartphone respondents to fading online survey technology. There’s mobile that fails and mobile that works.

MFour Mobile Research, Inc.’s aim since 2011 has been to define what mobile research can and should be, then create the new software and new approaches to panel-building that alone can make mobile work. Success means solving both ends of the equation: developing the right technology and recruiting and cultivating the right panel.

Developing The Right Mobile Technology

We’ve broken with all trappings of online research. Instead, we deploy technology that’s new to market research, the native app. Our proprietary app, Surveys on the Go®, instantly loads an entire survey into the respondent’s phone – including any pictures or multimedia content needed to enhance questions and answers. Embedding the survey into the phone is what makes it “native.”

Why does it matter? Because it frees respondents to complete surveys at their convenience. They don’t have to interrupt what they’re doing. They don’t need to be connected to the internet. Consequently, there’s no risk that the survey will become intolerably slow because of poor connections that lead to snail’s-pace downloads and data transfers. The survey can’t be dropped because of a lost signal.

At the opposite end of the spectrum are hybrid approaches that tether mobile devices to online survey software. A separate, back-and-forth exchange must take place for each and every question and answer. It’s a method that puts the respondent’s experience and the survey’s success at the mercy of internet connections that, as we all know, can bog down or disappear.

Essentially, mobile surveys embedded through a native app don’t have to be short and simplistic. Immune to smartphone signal issues, they can be long and sophisticated, and exploit special smartphone capabilities such as multimedia and Geo-location, which allows inviting panelists to surveys while they are still shopping or have just left a store. In our experience, app-based interviews run smoothly, regardless of location and even with interview lengths exceeding 20 minutes. Even at that LOI, we’ve experienced just a 6% drop-off rate. So much for the five-minute survey limit that’s commonly but wrongly posited for mobile research.

As for building a reliable, representative sample, good technology that begets a good respondent experience goes a long way toward drastically improving participation.

Curating A Winning Mobile Panel

With the right mobile technology, it’s possible to recruit the right kind of mobile panel. Ours numbers more than a million active respondents who take surveys solely on their smartphones and other mobile devices. They seem to like it, as reflected in strong ratings and comments at the App Store and Google Play. The mere fact that respondents can give us direct, unsolicited and very public feedback on their survey experiences makes app-based mobile a superior tool for becoming aware of panel problems as they arise – and taking quick action to solve them. It makes us accountable – as any firm that’s serious about its responsibilities and confident in its capabilities ought to be.

There’s much more to tell about the all-mobile approach – not least its ability to reach Millennials, Hispanics and African-Americans who, as GRIT notes, are vital to research but increasingly inaccessible to online surveys.

The Solution to Successful Mobile Research

But my main point is that the industry needs to understand that the available mobile technologies differ drastically. Then firms can make the natural comparisons, try different mobile providers, and see which can deliver a good panel and fast, reliable, representative data.

I think the most important sentence in the new edition of GRIT comes near the end, in a section called “Opportunities for the Market Research Industry” that examines ways forward from the current dead end.

“Mobile research has been seen as an opportunity for many years, but there is a sense that now we are at the stage where we can really start to exploit mobile data gathering techniques.”

Before you can exploit mobile techniques you must get to know one technique from the next. You have to stop stereotyping all iterations of mobile research as prone to the same limitations and drawbacks.

GRIT has done our field a great service by refusing to sugarcoat the sample problem and by sounding a clear alarm that something has to be done about it. There’s just one point I dispute.

I wouldn’t say that market research’s pervasive sample woes are “a known problem…with no solution in sight.” There is a solution, but until now it has been overlooked.

That appears to be changing. MFour’s year-by-year growth since we debuted our native app in 2011 suggests that an increasing number of researchers are starting to make the kinds of distinctions about mobile research that need to be made.

Market researchers need to do some research in their own backyards, to gain insights into their own most crucial interests – especially, as GRIT makes clear, when the industry’s health depends on it. Getting a more sophisticated understanding of mobile, market research’s most widely-adopted but least understood “emerging method,” would be a good start.