Lead up to MRMW: Interview with Paul McDonald of Google Consumer Surveys

 

As part three of the pre-MRMW interview series, I had the privilege of interviewing Paul McDonald of Google Consumer Surveys.  He gets into something quite near and dear to my heart: caring for the folks participating in the research through the process.  Without people that are willing to participate in research, we won’t have a future.  Here’s our conversation.

 

RM: What are you most excited to share about with us at MRMW and why?  What difference could it make for your audience if they were to implement what you’ll be talking about?

PM: One of the things we think a lot about here at Google is speed. There is a maxim we frequently use that comes directly from Google’s founders, “Faster is always better”. In the market research world, there seems to be a perception that speed comes at some cost, typically quality. It doesn’t have to be that way.

We think about speed in all aspects of our product and it’s really at the core of what we are doing. The initial idea for Google Consumer Surveys was conceived to find a faster way for users to access protected content online (on sites like the NY Times and WSJ that required you to pull out your wallet and pay to get access to that content). We had to find a way that reader could quickly get access and the publisher would still get value from the interaction. To be frank, we stumbled upon market research as the answer but once we realized that answering a question could take less than 10 seconds and that researchers would pay to get these questions answered we knew we were onto something.

With our large network of publishers we realized that not only could readers get access to content quickly, researchers could also get thousands of answers to questions in a matter of hours across a wide demographic.

It’s hard to put into words how large our respondent base could be. Every day on our ad networks we serve billions of ad impressions. If just a small fraction of our publishers implement Google Consumer Surveys we are talking about 10s of millions of unique users per day answering questions. Add in the volume from mobile applications and we could easily reach more than 100 million unique users a day. It puts us in a unique situation to let researchers choose to either get nearly instantaneous answers to their questions or to get a nearly perfect representative sample of a given population over a longer period of time, no matter how small that population might be.

So what does this mean for researchers? Among other things they can:

• Enable companies to make accurate, data driven decisions in near real-time
• Stop spending days or weeks getting the questions just right or making sure they’ve asked for exactly the information they need. Instead the data collection can be iterative, adjusting to the data collected to create the perfect survey.
• Track sentiment and opinion in smaller increments. A brand or politician can make messaging adjustments on a daily or hourly basis in response to feedback.

 

RM: It can be easy to bemoan the state of market research today.  Instead of us talking about what you’re against in the traditional MR space, I’d love to hear about what you’re for – what you stand for – in the MRMW space.  What makes this something you’re willing to stand up for?

PM: The respondent experience.

It’s easy to overlook the burden research puts on respondents. Instead of feeling empowered by seeing your feedback incorporated into products and services that you use and love, you end up cringing anytime you are asked to complete a survey. We’ve made a conscious decision to stand for the respondent first because ultimately we believe that the quality of the data you get back is direct reflection of the experience you put the respondent through.

By taking this approach we’ve had to make some tough choices for researchers. The premise of our ecosystem is that readers tradeoff their time for access to content. They need to be able to approximate how much time it will take to answer a survey in order to make that trade off so we intentionally limit the number of questions asked to any one respondent in one viewing to two questions. This results in accurate answers as it takes more effort to attempt to deceive us as it does to answer honestly.

We also limit the number of the characters in the question text and answer options to focus researchers on creating clear and concise questions. Similarly we limit the number of answer options show at any one time to make it easy for the respondent to comprehend the question and answer accurately. Finally we limit the question formats, purposely staying away from grids, complicated branching behavior and confusing rating systems. We’ve also introduced new question formats that are interesting for users and useful for researchers. Our image based questions are great for brand recognition tests, design and product comparisons. These questions are fun to complete and simple to understand.

Finally we review each survey for comprehension and adherence to our policies. So users are getting questions they understand and researchers are getting back data that is accurate and useful. In the end we think the respondent experience is the most important factor in quality research even if it means a more limited experience for researchers.

 

Editor’s note: You can learn more about the upcoming conference at the Market Research in the Mobile World website. Check out Renee’s blog at hellothereresearch.com.