Implications of Behavioural Economics for Research

By Neal Cole

Since starting out as a client-side researcher focus groups have frequently been recommend to me as a suitable exploratory method. Initially this wasn’t a problem. I was new to the industry. However, over the years I have gradually become more aware of their frailties. I have come across some excellent moderators. But I have also seen for myself how focus groups can be misleading and misused. The key problems I have experienced include:

• Respondents often don’t know what they want and tend to respond more favorably towards what is familiar to them rather than what might be new.

• Respondents sometimes say what they think you want them to say. I have observed the same respondent say completely contradictory things during a group. Seemingly to please the audience.

• Reactions to stimulus material is highly influenced by how the material is presented and communicated. This can be made worse if the moderator isn’t equally comfortable with all the variants.

• Respondents will over-analyze and rationalize topics (e.g. advertisements) when we are looking for their emotional response as this is often the key driver of the desired behavior.

• A single respondent may introduce new information into the discussion, and it can completely change the  direction of the discussion. Sometimes this can be useful, other times you wonder if it didn’t overly influence other respondents.

• One person can completely dominate the discussion and I have observed people changing their opinions to avoid conflict or because they mistake confidence for knowledge.

• Some respondents will not open their mouth once to contribute to the conversation.

• They are often conducted in a completely alien environment which does not match the context of the discussion. Viewing facilities can exasperate this problem as they can create a laboratory atmosphere.

• Discussion guides have too many items for discussion and include closed questions. I find moderators are generally overly optimistic about getting through a long discussion guide.

The industry may argue that most of these issues can be dealt with by an expert moderator. But we now have increasing evidence from behavioral economics that people are often irrational in their decision making. Humans rely heavily on mental short-cuts that allow them to make fast, automatic decisions that require little cognitive effort. This means that our conscious mind may not be aware of what drives a lot of our decisions. Such findings support the view that people post-rationalize what drives their decision making. This leads to people creating erroneous reasons for decisions when asked direct questions. Neuroscience also support the view that the majority of human decisions are made by this largely unconscious mind.

Indeed, it appears only a small proportion of decisions are made using our slower, more cognitive mind. This uses a lot more energy than our unconscious mind. For this reason we use this second system for complex, difficult tasks that requires a lot of thought. This has major implications for how we conduct and analyze market research. But what else can we learn from behavioral economics? Here are few key observations that have implications for focus groups and market research in general:

• Everything is relative and people don’t know what they want until they see it in context. Priming, anchoring, and framing are strong influences on how we respond to stimulus. People like to compare things that are easily comparable, and the context of how we present items heavily skews how we respond to them. This is especially problematic for anything totally new.

• Social norms such as reciprocation, commitment and consistency are powerful influences on our behavior. We are more strongly motivated by social rewards than by monetary rewards. And yet most participants in focus groups are incentivized  solely with cash.

• Our state of mind affects how we respond to questions. Dan Ariely’s experiments demonstrated how men’s responses to questions are heavily influenced by how aroused they are. This begs a question for market research. If we ask a respondent questions away from the natural context of the decision does this invalidate their answers? Context is an important aspect of how we respond to stimuli and we may not gather the key insights if we divorce the research from the natural decision-making environment.

• If uncertainty exists people look to the actions of others to guide them. People herd and like to do what other people are doing or saying. ‘Group think’ can also set in if the group is too homogeneous and the moderator seeks a consensus of opinion rather than having a show of hands. This is particularly worrying for focus groups.

• People have an extreme tendency to comply with the commands of someone in authority. If a respondent in a group comes across as something of an authority on the subject under discussion you may have a problem.

• Don’t underestimate the importance of presentation. It sets expectations for how people respond now and in the future behavior. This confirms my point about how people are very sensitive to how stimuli is presented. This may have implications for using mock up products or artwork rather than the finished design.

• First impressions count. Once a price has been established in our minds it will largely determine our perception of current and future prices. This indicates how people have a natural tendency to anchor prices from past experience and can be heavily influenced by priming.

Behavioral economics challenges the validity of relying solely on asking consumers direct questions, and highlights bias that can result from group dynamics (eg herding & group think).  The most fundamental challenge for market research is that humans are using two different cognitive systems for decision making. Thus, responses obtained from a person using their fast, efficient, unconscious mind will be invalid if in reality they normally use the more cognitive, conscious mind for that kind of decision. For evaluating advertising the reverse is likely to be the case. And what if they switch between the two systems at various points?

The consumer’s reliance on the unconscious mind for decision making and the unreliability of traditional research methods is analyzed by Philip Graves in his book Consumer.ology. He suggests that the most reliable methods for understanding consumer behavior are live tests and covert observation. Where practical I think this is excellent advice. For websites online experiments can be run relatively easily using A/B and multivariate testing. Where live tests are not feasible, there is strong evidence to suggest Predictive Markets, such as those developed by @BrainJuicer, offer us a reliable alternative.

For a number of reasons we are still going to want to ask consumers direct questions, though not necessarily in a focus group format. What behavioral economics suggests to me is that we need a framework in place to increase the likelihood that our methods and analysis will uncover the key insights. This means we need to consider factors including:

1. The mind system that is predominately engaged during the process.
2. The state of mind and context of the process being investigated.
3. How we present items to avoid priming, anchoring and framing bias.
4. Existing behavior – what habits and preferences currently exist.
5. Emotional responses that might be engaged
6. Social norms that may be influential

Under the right conditions I believe direct questioning of consumers can still provide useful insights. A good example is how Proctor and Gamble investigated the potential market for Febreze. Initially Febreze under-performed and was very close to being cancelled. To identify what was holding the product back P&G sent researchers into peoples homes to observe and record cleaning behavior. This allowed researchers to ask questions during or straight after cleaning episodes. Respondents were therefore in the right frame of mind and in a natural environment to discuss cleaning related products. What they discovered was that people were not aware of their own house odors. P&G’s strategy had been built around removing such unwelcome odors so this is explained why this wasn’t working. Further in-home observation and questioning identified how Febreze customers incorporated the product into existing cleaning habits. This was the break through they needed as they had been trying to create a new cleaning habit for Febreze. Instead P&G was able to piggy back onto an existing cleaning habit. This resulted in a completely new advertising strategy, and P&G has never looked back. Febreze now has annual sales of over $1 billion.

For focus groups, the dynamics of groups and the environment appears to make their findings more prone to bias. As Philip Graves points out:

“The artificial nature of the research environment can also be responsible for not flagging up something that, in the real purchase environment, is unconsciously reinforced and hugely significant in determining a product’s fortune.” Philip Graves, Consumer.ology

Jonathan Ive, Apple’s Senior Vice President of Industrial Design, once said that Apple had found good reason not to use focus groups because:

“They just ensure that you don’t offend anyone and produce bland inoffensive products.”

Does this mean that the focus group is dead. Given their popularity I suspect not. Further, recent experiments with using gamification in focus groups have shown how we can use this technique to disrupt and counter dominant group members. What this suggests is that we should look to incorporate some of the insights from behavioral economics into the design of focus groups. Why not try running a focus group in a kitchen environment if the topic is about cooking appliances? Use role play and gamification techniques to bring out the appropriate state of mind.  Observe body language and facial expressions, not just what people say. I know some moderators already use these types of strategies and perhaps we need to promote them more heavily to clients who insist on using focus groups. Probably the most important point is that we recognize the limitations of our research tools and use behavioral economics to help us avoid certain traps and interpret findings in a more scientific way.

Thanks for reading. I hope this blog has challenged a few myths and generated some useful ideas.

Further reading: Influence by Robert B. Cialdini, PHD

Predictably Irrational by Dan Ariely (@danariely)

the Upside of irrationality by Dan Ariely

The Wisdom of Crowds by James Surowiecki

Consumer.ology by Philip Graves (@philipgraves); Nudge by Richard Thaler (@R_Thaler).