7 Points To Consider Before Using
The Net Promoter Score (NPS)

1. The NPS is not diagnostic.

Although the NPS score may suggest you have a problem, the score alone doesn’t tell you what needs to be fixed.   To the extent that Reichheld acknowledges this situation, he suggests that it can be addressed by asking an open-ended follow-on question asking why the respondent gave the rating in question.   While such responses may provide some insight into the nature of the organisational and customer issues faced by the company, they are anecdotal at best and tend not to lend themselves to the level of analytical rigor needed for effective problem solving and setting priorities for improvement planning.

2. The division of respondents into the categories of promoters, neutrals and detractors is arbitrary and has no scientific basis.

This fact by itself robs the NP score of any objective meaning.  How is it that a one-point different in score on an eleven-point scale can accurately determine whether an individual is a promoter rather than a neutral, or a neutral rather than a detractor, whatever those terms mean?  “The rule-of-thumb score classes proposed by Reichheld (promoters are those respondents who give a likelihood of recommendation of 9 or 10 while the detractors give 6 or less) are not supported statistically, mask important changes and potentially mislead management that there is negative NPS when this may not be the case.” – Ken Roberts, Forethought Research Australia.

3. Cultural differences can invalidate comparisons of NPS results.

Members of some cultures tend to avoid providing extreme scores on surveys.  For example, respondents from Japan, Australia and some northern European countries are far less likely to select “9” or “10” when compared with American respondents.  Thus, NPS results for these countries will appear to be lower even when, perhaps, the respondents are actually more favourably disposed than their counterparts in other cultures.   To account for this, some analysts have suggested including “8’s” as promoters, but this is still rather arbitrary and allows lots of room for error. 

4. The wording of the NPS question is questionable.

“How likely would you be to recommend…?” is a question about future intention with the implication that the question is behavioural. Yet a large body of research indicates that claimed intention is a better reflection of present attitudes than it is of future behaviour (Bird, Ehrenberg and Barnard).  In addition, some who read the question literally may respond with a low score while feeling a high degree of loyalty simply because they think they may have few if any opportunities in the future to recommend the company.  Most respondents in B2B surveys have no say or influence in purchase decisions.

The Net Promoter question presupposes a positive response. For example, “How likely would you be to refer a friend” assumes from the outset that you would be at least somewhat likely to recommend. Studies have shown that wording questions differently can lead to very different results for “favourable” and “unfavourable” responses. (Aapor.org. 2006-2007. American Association for Public Opinion Research. 2 February 2008.)

For these reasons, we prefer to ask how willing the respondent is to recommend, a question that is plainly attitudinal and does not pretend to be behavioural.  This phrasing reduces ambiguity for the respondent.

In addition, the NPS question itself is unipolar (likeliness to recommend) but Reichheld treats it as bipolar (likely to detract vs. likely to promote).  The implications of this are unclear but may reduce the validity of the results.  Using our recommended wording avoids this trap.

5. The single NPS question is less reliable than a composite index would be.      

Researchers and statisticians generally agree that composite indices are more stable than individual item scores.  Some have suggested that ratings of customer satisfaction and intention to repurchase might be added to the recommend question to improve reliability.  “In his Harvard Business Review article ‘The One Number You Need to Grow’, Reichheld maintained that since his tests showed propensity to recommend to be the single question that had the strongest statistical relationship to future company performance, there was no point asking any other questions in customer surveys… (However) a single item question is much less reliable and more volatile than a composite index.” – Customer Satisfaction – The customer experience through the customer’s eyes,” – Nigel Hill, Greg Roche and Rachel Allen.   

6. The eleven-point scale used by NPS is problematic.

By collapsing the responses into three groups (0-6, 7&8 and 9&10) much information is ignored.  Using a 3-point scale (1, detractor; 2, neutral; 3, promoter) would be the equivalent scale and would provide the same information.  However, there is no reason to believe that someone scoring the company with a zero would have the same attitude and customer behaviour as someone scoring a six.  This is nonsense on the face of it, yet NPS does not account for these obvious differences which, in fact, it measures but doesn’t use.

Additionally, many psychometricians have suggested that the average respondent can comfortably discriminate among no more than seven points at a time.   Thus longer valued scales may pose difficulties.  This in part may be why the NPS 11-point scale has been shown to have lower predictive validity than other scales.  The paper “Measuring Customer Satisfaction and Loyalty: Improving the ‘Net-Promoter’ Score” by Daniel Schneider, Matt Berent, Randall Thomas and Jon Krosnick demonstrates that the 11-point scale has the lowest predictive value of any of the scales tested. The authors recommend a 7-point scale with labelled ends and midpoint for the NPS question.   The authors also recommend a bipolar scale for a reworded variant.

7. The original research behind NPS has been called into question.

In 2007, other researchers began trying to replicate Reichheld’s results and failed to do so.  In particular, Timothy Keiningham at Ipsos Loyalty and a group of researchers published a study in the Journal of Marketing that found relatively low correlations between NPS and a change in revenue. The researchers claimed that “Net Promoter in no way would be categorised as the ‘single most reliable indicator of a company’s ability to grow.'” (Keiningham, Timothy L., Bruce Cooil, Tor Wallin Andreassen, and Lerzan Aksoy. “A Longitudinal Examination of Net Promoter and Firm Revenue Growth.” Journal of Marketing 71 (2007): 39-51.)  Keiningham would later claim that support for NPS was based largely on anecdotes with no scientific basis to them. 

In a Marketing Management article the same year, Dr. D. Randall Brant stated that NPS “isn’t always the best predictor of customer behaviour or business results” and claimed that Reichheld himself admitted that the question was “simply irrelevant” in some markets and sectors.  (Brandt, D. Randall. “On the one number you need to grow, one size doesn’t fit all.” Marketing Management 16.1 (2007): 20-25.)

In response to these issues, we suggest the continued use of the recommend question, but to substitute willing to recommend in place of likely to recommend.  We believe that a 7-point scale is optimum for this question and that the 11-point scale should be avoided.  Finally, we recommend against the use of the NPS framework, using the recommend question as the outcome variable in a driver analysis or as an element with other factors in a loyalty scale.