One Final Time – NPS is Not a Lead Indicator of Growth!

2023 marks the 20th anniversary of Reichheld’s ‘The One Number You Need to Grow.’¹  I find it astonishing that businesses implement NPS programs with the expectation of growth.  Astonishing because of the constant stream of objectively measured, empirical evidence that, as a lead indicator of growth, achieved through either retention or acquisition, NPS has close to zero efficacy and no predictive validity whatsoever.

To be clear, NPS is not a non-financial lead indicator of growth.  It is not now, nor was it ever, the one number you need to grow.

How long have we known Reichheld was wrong?²

The advocacy question was devised by Forethought in 1997 as a dependent variable for modelling the drivers of advocacy in business-to-business markets and was later applied by Reichheld as a categorical question for identifying the proportion of Promoters (9 & 10) from Detractors (0-6).

NPS has close to zero efficacy and no predictive validity whatsoever

Six-years post Forethought applying the question in dozens of longitudinal tracking studies Reichheld published his ‘The One Number You Need to Grow.’  Implicit in Reichheld’s ‘Grow’ claim was change in relative market share.  In 2006 in the ESOMAR publication ‘Research World,³’ Forethought released a 9-year study of the best performing non-financial lead indicators.  Forethought conclusively found that advocacy was inferior to all other non-financial lead indicators tested such as value-for-money^4,5 in explaining market growth.

In July 2007 in the first peer reviewed, academic journal article by Tim Keiningham⁶, the authors also made the conclusive finding that ‘The research fails to replicate his [Reichheld’s] assertions regarding the “clear superiority” of Net Promoter compared with other measures in those industries.’

Why didn’t the science-based articles stem the unbridled adoption of NPS?   In a nutshell, the relatively silent science was pitted against innumerable, exuberant consultants selling the simplicity nirvana coupled with the herd instinct of organizations and the rise of the NPS fundamentalists.   NPS fundamentalism is when without question, and to the exclusion of all scientific rigor, one believes that improving an NPS will lead to better business outcomes.

The relatively silent science was pitted against innumerable, exuberant consultants selling the simplicity nirvana

If not NPS, then what is the lead indicator?

Organizations seek to change the trajectory of their brands.  The purpose of that change is ultimately to raise profitability or market share.  These outcomes are the organization’s lag indicators.  In this pursuit, what is paramount is to identify the non-financial lead indicators and to focus on the drivers of these lead indicators.  The following exhibit illustrates that point.  The desired outcome in this example is to gain market share.  The lead indicator is a dependent variable that will yield a predictive model of what drivers the organization must focus on if it is to gain relative market share.  Continuing the example, the initiative is the advertising that addresses the drivers.

Whether the lead indicator is retention with the initiative focused on addressing operational performance or acquisition with the initiative focused on marketing communications, advocacy or any categorical abomination of the advocacy construct (Detractors, Promoters), are not lead indicators of growth.

I say that as an absolute statement however, for a tiny minority of intermediated services such as business-to-consumer broking of financial services, advocacy can be a lead indicator.  For example, the advocacy scores of consumers with recent, episodic contact with a mortgage broker is a lead indicator to that brokers’ prosperity.  That exception is so isolated in the context of the “mass-marketing” of NPS that I feel justified saying again, advocacy is not a non-financial lead indicator of growth.

The logic is as simple as it is compelling.  There must be an explanatory relationship between the lead indicator and the lag indicator.  Retention and acquisition are lead indicators of brand growth.  I can hear the orchestra of consultants screaming, “yes but, advocacy is a lead indicator of retention and acquisition.”  Nope!  Retention is explained mostly by operational performance.  Acquisition is explained mostly by value (price and quality) for money.

There must be an explanatory relationship between the lead indicator and the lag indicator

What is more, there are two dangerous paradoxes with NPS.  When Detractors defect, your NPS goes up and when you gain share, your NPS goes down because new customers have lower advocacy scores.

Let’s hear from science

Forethought examined 30 data sets across six categories (superannuation, supermarket, consumer goods (cereal), quick service restaurant, banking, and aviation).  We selected these 30 data sets because in each instance, we had access to corresponding independent, third-party data of objective, in-market performance.

What we found is that NPS performs poorly as a lead indicator of both current market performance and changes in market share.  At a practical level, what this means is that you can be investing in improving NPS and not see a corresponding increase in market share.

An alternative lead indicator that was tested was Brand Health.  Brand Health is the weighted performance of a brand on the rational and emotional drivers of brand choice (at Forethought this is referred to as Prophecy Thoughts & Feelings®).  The following two exhibits speak for themselves. Note the contrasting R2.

You can be investing in improving NPS and not see a corresponding increase in market share

The following exhibit is a Brand Health model of consumers deciding between banks’ transaction accounts.  ‘Recommended by peers, friends and family’ is a minority driver – just 2.1% of choice.  Little wonder advocacy is not a reliable lead indicator of gaining market share.

Every promotor is a misallocation of resources

If NPS remains your religion and you are striving for more Promoters, then you are pushing more customers into a place that makes little economic sense.  If you must use NPS, then your primary objective should be to completely eradicate Detractors (0-6).  Perhaps counter intuitively, economically speaking, the optimum NPS is zero made up of no Detractors and no Promoters and 100% of customers scoring your organization seven or eight.

The optimum NPS is zero made up of no Detractors and no Promoters

Putting aside that NPS is not a lead indicator of growth, the effort organizations waste in the pursuit of more Promotors is a misallocation of scarce resources.  The cause of this waste is the flawed conclusions concerning the inflection point on the eleven-point scale.  This is illustrated below using advocacy scores for a large financial service.  The same distribution is repeated in many hundreds of data sets.  On the scale, the point of diminishing returns is eight and not nine.  The Promoters do relatively little for defection and therefore, the more Promoters you have the more resources you have wasted.

Assessment of the NPS menu

NPS has evolved into a menu of services (read revenue opportunities for consultants).  These include strategic, interactive, episode and relationship NPS.  The following table provides a brief assessment of each.

 

Type of NPS	What is it and does it help?
Strategic  NPS	Looking at performance across a set of competing brands.  Regardless of having recent experience with the brand, customers score their main brand on their likelihood of advocating that brand.  Comparisons are drawn between the scores each brand receives from its own customers.  This view is fundamentally flawed.  Customers assess prospective brands relative to their incumbent brand.  How others assess their own brand is largely irrelevant in acquisition or defection.  If growth is your objective, then strategic NPS is an utter waste of effort.
Interactive  NPS	Recent contact survey focused on a single interaction.  Of limited value for driving growth when the interaction has not been validated as explaining the genesis of growth.  That is, changes in the business outcome, defection or acquisition.  Therefore, initiatives designed to lift performance are likely to be too numerous and better directed elsewhere.  Often the advocacy question is accompanied with an open-ended reason for the score.  Stated reasons for behavior is considered vastly inferior to inferred reasons established from modelling.
Episode   NPS	Recent contact survey following fulfillment of a need.  Of limited value because the episode might not explain changes in business outcome such as defection or acquisition and therefore, initiatives designed to lift performance may be better directed elsewhere.  The episode or “need” is often an organizationally defined process.  For example, the mortgage application is a bank process not a consumer need.  The need is more likely to center on improving the living conditions for the family.  The emotional motivation might be love and pride.
Relationship  NPS	Specifically looking at a single cohort of customer.  Regardless of having recent experience with your brand, customers score the brand on their likelihood of advocating that brand.  If retention is your objective then, Relationship NPS is waste of effort.  Of far greater value would be using a dependent variable that accurately predicts defection.  Advocacy is not that construct.  Being unwilling to advocate a brand is not the same as intending to defect from that brand.

Gaming NPS

The elephant in the NPS room is the rampant gaming of data collection.  Despite the absence of evidence of the veracity of NPS, management have KPI’s and incentives based on NPS.  On one hand, improving the score will not advance the cause of the organization but on the other, it might serve to benefit the individual manager thus contributing to the vicious circle of NPS being gamed and the continuing investment in NPS.

The advantage to management is not just incentives related.  Initiatives designed to improve NPS can provide employment to groups of colleagues despite there being little objective benefit to the growth aspirations of the business.  Organizations need to have an executive led process for reviewing and killing initiatives not materially addressing the objectives of the organization.  Few organizations have a rigorous approach to assessing the veracity of initiatives.

Organizations need to have an executive led process for reviewing and killing initiatives

I have many examples of gaming NPS.  Color coding the scale changes the scale from the intended scale with the zero and ten having semantic anchors (highly unlikely to recommend to highly likely to recommend) to coaching the respondent to take the organization’s orientation.  Green is good.

The second example is a bank relationship manager providing their business card with guidance to using the scale scribbled on the card which is dramatically different to the original intention of the scale.  Seven to eight is given the semantic anchor as “bad.” Nine and Ten is “good.”

All in all

Yes, NPS is based on one simple question and in terms of an accessible measure with a market orientation that engages the front line and is universally comprehended, NPS programs maybe a useful managerial approach.  However, the topic here is specifically, lead indicators of growth of customers and not front-line employee engagement.  Managerially, this is a critically important distinction.  If the organisation’s objective is growth, then NPS is the wrong KPI. 

Our analysis reveals that there is an expansive naivety about setting out to improve NPS and hoping that a business outcome will follow.  Executives need to demand to see the in-market evidence of the relationship between NPS and in-market, objectively measured performance.  Executives who make such a demand, will most likely receive correlations with NPS and in-survey attitudinal questions which do not translate to in-market behaviour.

To be crystal clear, NPS is an inferior non-financial lead indicator and to make matters worse, the pursuit of more Promoters is most likely diminishing the optimum economic returns to its shareholders.

 1.  Reichheld, F., The One Number You Need to Grow, Harvard Business Review, December 2003

 2. This section of the paper has benefited from a previous piece – Roberts, K., Hansen, K., How likely would you be to recommend NPS? Forethought Client Briefing, February 2018

 3. Roberts, K., Value Speaks Louder than Words, Research World ESOMAR, December 2006

 4. Gale, B.T., Managing Customer Value: Creating Quality and Service That Customers Can See. The Free Press, New York, 1994

 5. Please keep in mind that the term “value” is commonly used as meaning low price however, the use of the term here is the economics definition of “value – what is received for what is given.

6. Keiningham T et al, A Longitudinal Examination of Net Promoter and Firm Revenue Growth, Journal of Marketing, July 2007