A few years ago I was presenting the results of a B2B customer survey.  It provided surprising insight into how customers were using a service provided by my client.  The survey was a great success, enabling us to refine campaign creative and messaging, build campaigns on the insight and compare self-reported usage to actual usage.  What was the first question I was asked at the end of the presentation?  What was the Net Promoter Score?  

Frederick Reichheld wrote about the Net Promoter Score (NPS) in Harvard Business Review in 2003.  It has spawned an industry complete with a website, certification courses, conferences and papers for and against the methodology.  The basic idea behind NPS is that growth by an organization, brand, or product is correlated with the percentage of its customers who would recommend them.  

Per the HBR article, it requires the following survey question to be asked: “How likely is it that you would recommend [brand or company X] to a friend or colleague?”  The possible responses should range from zero to ten.  Zero means not at all likely to recommend.  Five is neutral.  Ten means extremely likely to recommend. 

This is where my friend’s survey question went wrong.  As per my earlier post, both of the NPS questions used a scale from 1 to 10.  With an even numbered scale (in this case 10 choices), there is no true midpoint.  In addition, I noted affect of the words used to describe endpoints of the scale earlier. 

While many marketers I know have heard of NPS, they are less familar with the research challenging the NPS methodology.  If you are interested in learning more, here is a recent article from Quirk’s Marketing Research Review available from Business Over Broadway.

The Net Promoter Score (NPS) means many things to many marketers but it starts with a single survey question about the interviewee’s likelihood to recommend an organization, product, service, etc.  To calculate NPS, you subtract the percentage of detractors from the percentage of promoters.  The idea was first published in Harvard Business Review but you may have read about it in Advertising Age, Journal of Marketing or any number of publications. 

A friend asked for my thoughts about NPS recently because he was getting two different results for NPS depending on how he asked the question.  One question asked, “On a scale of 1-10, how likely is it that you would recommend company X to a friend or colleague?”  The scale ran from 1 (Not likely) to 10 (Very likely) with 5 described as Neutral.  The scale on the second question went from 1 (Definitely would not recommend) to 10 (Definitely would recommend).   No qualifiers were given for numbers 2 to 9 on the scale.

I correctly guessed which question produced higher results.  Can you?  In my next post I will write more about NPS because both questions were wrong. 

However, I would like to focus on the art of writing survey questions.  The only difference between the two questions are the scales used.   The second scale was more extreme than the first.  “Definitely would not recommend” is much stronger language than ”Not likely”.  The first scale also highlighted position 5 by using the word ”Neutral” instead of the number.  All the other positions on the scale had numbers.  This had three consequences.  First, this made it appear that 5 was the midpoint.  However, On a scale from 1 to 10, the midpoint would be 5.5.  Second, by replacing the number 5 with a label, it calls attention to the middle of the scale and could skew answers as a result.  Third, position 5 should not have been described as neutral when in fact a 5 is considered a detractor according to the published methodology. 

This real life example reiterates how difficult it can be to write good survey questions.  How you ask the question can determine the answers you will receive.