Voice Of The Customer Surveys – Useful Or Fundamentally Flawed?

Voice of the Customer (VOC) surveys are a popular method for organisations to monitor customer preferences and levels of satisfaction. However, how robust is the standard process of interviewing a few customers and then developing and administering a highly structured survey?

The VOC survey framework appears to be based upon an outdated and false view of people as rational, independent agents with relatively fixed preferences. Because of the way the human brain works people have limited access to the emotional, social and psychological motivations that drive much of our behaviour.

Surveys also ignore the power of contextual influences which means that VOC programmes are highly likely to generate unreliable and misleading results which contribute to an illusion of understanding customers.

A flawed measure:

• Asking direct questions is unreliable because many of our daily decisions are made without much conscious thought and we post-rationalize decisions to ensure consistency with our internal model of the world. When people answer direct questions they attempt to rationalise their decisions and behaviour. They naturally construct a narrative that explains their  actions in a rational and consistent way.

• However, this fails to capture many of the underlying motivations of human behaviour. Further, as our brains use mental short-cuts to save energy and speed up decision making, we are prone to making sub-optimal and often irrational decisions. This is contrary to how we like to perceive ourselves and naturally we don’t articulate this when answering survey questions.

• Asking people about importance is especially misleading. Psychologists have found that relative importance of an item is heavily influenced by the ease with which we can retrieve it from our memory. This often correlates heavily with the amount of coverage an issue gets in the media. I noticed this when I managed a survey of financial advisers. The importance of financial strength always shot up after media coverage of any kind of financial crisis.

• Our ability to recall an event is also limited as a memory is constructed from a series of brief fleeting moments from an experience. We construct a memory from visual snapshots, thoughts, feelings, smells and sounds. The perception of an experience is also influenced by range of factors including our mood, social context, our vocabulary, the physical environment and our knowledge.

• However, the reconstruction process itself is also the product of these same factors. This means that each time we recall an event we have to piece these elements back together and inevitably our memory changes each time we recall an experience. Indeed, we frequently have false memories of events that we will passionately defend as our brain attempts to retain our internal consistency with our values and beliefs.

• Our memory is also biased towards  remembering what happened during the most intense moment of an experience, and what occurred at the end of the episode. The length of the experience appears to have little impact on our overall satisfaction with an event.

• Our herd instinct means we are heavily influenced by what other people in our network and beyond are doing. We are constantly copying people, especially when we find ourselves in a new or uncertain situation. Social norms and trends are also powerful forces, but again we may not  be consciously aware of these influences and so we cannot expect people to articulate this via direct questioning.

Analysis and Action:

VOC surveys also fail to deliver when it comes to analysis of results as little or no allowance is often made for the fundamental flaw in using direct questions to obtain the data.

• Due to the survey design operational and customer facing areas often find VOC feedback to be too generic and therefore not actionable. They want to drill down to specific locations or touch points but sample size are usually not sufficient to allow for such analysis.

• Unfortunately research executives can sometimes be pressured into providing analysis based upon tiny sample sizes. This gets quickly circulated around the organization to support changes in service or product delivery. The fact that responses are unlikely to be an accurate reflection of real customer motivations and preferences is usually ignored until results contradict what a senior stakeholder wants the data to say.

• Sponsors like to set targets to improve VOC scores but ratings tend to fluctuate for no obvious reason. When expensive changes to the product or service are implemented is it surprising that we may not see any change in customer satisfaction?

• Some organisations now base changes on an understanding of our selected memory bias by focussing on the peak and end moments of an interaction. This may lead to improved VOC scores but does this really mean that we are delivering an improved customer experience?  Are we not just kidding ourselves by using human psychology to temporarily boost a flawed score?

Experiments and Observation:

Top retailers have known for decades that if you want to find out if something new works running a controlled test in a number of stores is more reliable than asking people direct questions. This was the original A/B test and is the reason why online retailers are now some of the biggest users of this experimental research design.

“Just following consumer wishes leads to replaceable products, copycat advertising, and stagnating markets.”
Stephen Brown, Professor at the Kellogg School of Management

Direct questioning of customers is limited for the reasons given above. However, VOC surveys are particularly problematic because the standard framework developed as part of the Six Sigma methodology gives it an aura of validity that the technique does not merit. This is an illusion which Six Sigma followers espouse due a lack of understanding of basic human decision-making and psychology.

More emphasis on conducting experiments, together with observing or listening to real customer interactions would be more effective methods of research. Co-creation can also be a powerful approach to allow brands to open up a two way conversation with their customers. Direct questioning for VOC programs needs to be used sparingly and the results treated with extreme caution.

Thank you for reading my post and I hope you will find some of my other post of interest.

Recommended reading:

Consumerology: The Truth about Consumers and the Psychology of Shopping (new revised edition, including a new preface from the author)<