When Marketing Research is Like a Sunset On Pluto

 

By Joel Rubinson

Gregory Berns, distinguished neuro-economist says, “the brain is fundamentally a lazy piece of meat”.

He demonstrates the principle by asking us to imagine a sunset on a beach and then to imagine a sunset on Pluto.  He asks us to note how much harder the latter is to do.  Also, while you get answers, they are highly inaccurate.  So, when is marketing research like asking people to imagine a sunset on Pluto?

Recently, I have seen a report on people’s prediction of their comfort with the idea of stores becoming just showrooms a number of years from now where people actually make their purchase on their smart phone. Really? Consumers can predict the future like that? That is just one of many Pluto studies I see regularly reported on in the trade press.

Behavioral economics studies the shortcuts we take to simplify our decision making when faced alternatives for which we have varying degrees of incomplete knowledge and gives us clues as to how surveys can take a really wrong turn.  Think about how someone fills out a survey when some of the questions don’t quite work.  They are making choices about how to answer questions and the less natural the question or harder to recall the right answer, the more they are using short cuts that are predictably irrational, as Dan Ariely might say.  For example, when people are asked which brands of cereal, they bought in the past three months and we compare answers to actual tracked behavior, we see significant overstatement of brand penetration for every brand.  If I first ask about the past year, past six months, and then the past three months, I will get much closer to the truth.  What cognitive and emotional processes are leading people to access their memory in a faulty way resulting in what is called “telescoping”? How about private label brand preferences? They are always understated.  People buy private label brands much more than they admit in a survey.  Why and what can you do about it?

Think about brand attribute ratings batteries.  Every tracker and A&U has them.  What happens when someone is responding to a phrase that does NOT describe how they personally think about the brand or even choices in the category?  They guess at the answer!  They go through a brief self-analysis, translating the way they DO think into an answer for the question as asked.  The respondent level test-retest reliability of such answers is really low (similar to why 50% of respondents often do not get classified into the same attitudinal segment if the questionnaire is re-administered).

Here are seven tips for avoiding marketing research about Pluto

1. Don’t ask questions about behavior if you can measure behavior directly. This is especially true today when digital and shopping behavior is so easily tracked.
2. Ask questions with natural vocabulary, revealed by social media listening.  How do people naturally talk about your brand, category, and need states? Change your questionnaire and attributes to reflect emerging vocabulary and don’t worry so much about preserving trends on irrelevant phrases.
3. Only ask people to recall ephemeral behaviors (like shopping, or using a smart phone) right at that moment
4. Make the survey answer choices mirror the real life purchase or consumption choice you are trying to research as much as possible.  That is why constant sum approaches do a better job at predicting an individual’s probability of buying a brand than 5 point scale purchase intent questions. In real life, shoppers choose among brands, not purchase intent answers for a given brand (as Eric Marder has noted in his book The Laws of Choice).
5. Nudge (as in the behavioral economics book) respondents’ answers about private label or anything else that is repeatedly underreported.
6. Keep surveys short.  A series of hard questions will have a cumulative effect at depleting a respondent’s energy and make their “guesses about themselves” even worse as the survey goes on.
7. Use lead-ins and warm-ups so respondents know what you are after. While this has been frowned on by many purists as potentially biasing, it actually anchors the respondent by giving them a framing, points of comparison, and categorization to work from.  It gets their head and heart into the moment that the research refers to, which according to George Lowenstein another noted behavioral economist, is critical. People want to be helpful; equip them to give you the right answers!

Marketing research needs to be mindful that behavioral economics principles apply to the dynamics of survey taking.