An alternative method of reporting customer satisfaction scores

Though customer satisfaction evaluations are widely used, reporting of these scores has varied from one study to another. This is likely the result of each method’s advantages and disadvantages, as well as the personal preferences and habits of the researcher. We recently had the opportunity to report customer satisfaction scores in a unique format that assimilates the advantages of various methods and provides the manager with a clearer picture of where to take action. In this article we review various reporting methods and outline our method with an example. Further, we also discuss a type of reporting that is becoming increasingly common especially in the health care arena, i.e., the issue of comparing the performance of various facilities or centers that belong to a single network or organization. We show how our method can be applied for this purpose and why it is advantageous.

 

Current reporting formats

First, consider the prevailing methods for reporting attribute satisfaction scores.

• Mean scores
• Top two box scores
• Top box scores
• Top two box and bottom two box scores

 

(Whether it is the top two box or top three box score that is reported is irrelevant because the principle remains the same.)

The advantage of reporting the mean is that it is a summary score that takes into account the frequency of answers for each scale point. As often found in practice, the disadvantage is that the manager doesn’t see much difference between the mean satisfaction scores on various attributes.

For example, on a seven-point satisfaction scale, the mean satisfaction scores on many attributes tend to be clustered in the five to six region. Thus, even though some of them may be statistically different from others, for a manager, it is not easy to understand where efforts need to be focused to improve overall satisfaction. Further, means of attributes which have bimodal distributions (high top two box and bottom two box scores) could have the same mean as those with normal distributions.

Reporting top two (or top three) box scores has the advantage of providing more variation in the data. Thus, it is easier to identify the attributes where the company is performing poorly. The disadvantage with a top two box score is that it is often quite high. This sometimes leads to complacency because its high value seems to indicate that respondents are very satisfied. However, reality could be quite different because the measure doesn’t take the full distribution into account. Consequently, even though the bottom box scores may be different, two attributes with equal top two box scores could be considered to be equal.

Top box score has been suggested more recently as a better way of reporting satisfaction scores. It includes only "totally satisfied" customers and hence the lower value associated with a top box score discounts any notion of managerial complacency. However, as in the previous case, top box scores ignore the rest of the scale, potentially masking trends in bottom box scores.

To overcome this problem, in some cases researchers report both top two box and bottom two box scores. While this provides a more complete picture, it forces the manager to integrate two pieces of information for each attribute. This is especially hard when there are many attributes included in the study and, more important, when scores are compared across many attributes and locations.

 

Alternative format

The solution we developed for this situation was to create a single statistic which utilized both top two box and the bottom three box data from a seven-point scale. More specifically, we subtracted the bottom three box score from the top two box score to provide a single rescaled score that not only varied between attributes but also took both ends of the distribution into account.

Having constructed this new score, we took it a step further. Rather than just subtract the bottom three box score, we subtracted twice the bottom three box score from the top two box score. (Refer to the example below.)

This formulation has several implications. Marketing studies conducted by us and other researchers have shown that there exists an asymmetric effect on satisfaction. That is, the impact of negative attribute performance on overall dissatisfaction is higher than the impact of positive attribute performance on overall satisfaction. By using twice the bottom three box score, the formulation proposed here takes asymmetry into account by saying that a customer in the bottom box hurts a company more than the gain provided by a customer in the top box.

It also means that moving a customer out of the bottom two box is harder than moving a customer into the top two box. Therefore, the former achievement should be rewarded more than the latter. Because the bottom two box score is weighted to twice its value in this formulation, moving customers in and out of the bottom boxes has a greater impact on the scaled attribute score than moving customers in or out of the top boxes.

The number of scale points to be included in the top and bottom boxes is dependent on the scale used in the study and the distribution. Similarly, the extent to which the bottom boxes should be weighted (twice, thrice, etc.) is dependent on the particular study. But the principle remains the same. A single score is obtained that has more variation than the mean score, includes both ends of the scale and is weighted to include the asymmetric effect.

 

This article was written by Rajan Sambandam and George Hausser of TRC, a full-service market research provider located in Fort Washington, PA.