On NPR’s Morning Edition, Jan 8, 2019, social science correspondent Shankar Vedantam, discussed an issue that should be top of mind for any company that uses 5-star rating systems.
According to NYU professor John Horton who researched online marketplaces, most customers give their Uber or Lyft driver a 5-star rating—regardless of the quality of their ride.
For anyone who hasn’t used a ridesharing app, once you’ve arrived at your destination, the app asks you to rate your driver between 1 and 5 stars. These ratings are said to provide valuable information, but if customers aren’t sharing what they really think what’s the point?!
John Horton’s Explanation
John Horton’s explanation of why customers don’t share their honest opinions is they worry their feedback could be used to punish drivers. Few customers want to damage the livelihood of a driver who helped them, even if their car did stink to high heaven!
Horton explains it’s a form of altruism in that the customer is concerned for a stranger. But he adds that the most altruistic customers are those who give honest feedback, thereby improving the rides for everyone.
HappyOrNot
Another system that’s bound to produce skewed results is the now ubiquitous HappyOrNot. This company (and similar kiosk systems) is the latest twist on customer feedback where customers pick one of four smiley faces to rate their satisfaction. There are countless flaws here but certainly implementation is an issue. No one wants to give an associate a frowny face when they’re standing in front of you or nearby.
Whether stars or smiley faces, there is a reasonable case to be made for altruism inflating ratings. But, there are three deeper reasons why customers’ ratings can widely diverge from their actual experiences.
Reason 1: Customers Fear Repercussions
With big data and social media, there is the perception that we are all highly connected and visible. While online marketplaces may say the associate can’t see your ratings, this may not seem true to customers.
In fact, sometimes, it’s hard to see how it could be true. For instance, if you are the last ride of the night and give a low rating, that driver may easily guess where that came from. The prospect of an Uber Driver knocking on my door (where he just dropped me off) to complain angrily about his rating is terrifying (particularly as a woman). It would be interesting to see whether Horton investigated the effect of gender on these types of ratings.
Beyond personal safety repercussions, if a customer wants to continue using a company, they may fear their future interactions will be tarnished if they award a sad face or a 1-star rating.
Reason 2: Not Everyone Has the Same Standards for 5 Stars
Customers who are new to Uber or Lyft may not understand what a high-quality ride can mean. They may give a 5-star rating on their initial rides because being delivered to their destination in one piece is the only expectation they have.
However, thirty rides later, when they finally have a true 5-star experience, (pristine car, efficient ride, pleasant or no conversation) customers will have a different standard. Without clear guidelines from a company about what a 5-star rating means, there is bound to be huge variation.
Reason 3: Customers Don’t Know What They Don’t Know!
With Lyft or Uber, we’re sometimes asked to rate our drivers before we know something was off. For example, if a customer is visiting a new city, they may not know that their 45-minute winding scenic route should have been a 15-minute drive. It might not be until the next day that it becomes clear it wasn’t normal. Customers don’t have the opportunity to change their rating or give feedback retrospectively and that skews data.
This same issue played out with our IT company recently. After assuring us our problem was fixed, they prompted us to fill out a customer feedback form. Since they said they had fixed the problem, of course we gave them 5 stars! But the next day, it turned out the problem was not resolved at all. By then, the ticket was closed. The company thinks I had a 5-Star experience, but I certainly did not! Ugh.
How to Get Honest Customer Feedback
Recently, I wrote How to Invite the Most Honest Customer Feedback. That post provided a few guidelines for how companies can get more honest survey feedback. Here are a few more suggestions that apply specifically to starred and emoji rating systems.
Prove Anonymity: Brainstorm and think of some way to prove to customers their responses really are anonymous.
Consider WHEN you ask for Customer Feedback: You’ll get the most accurate data when your customer has had a complete, end-to-end experience. In my example with our IT company, it would’ve been better if I had rated them AFTER I’d had a chance to see if their solution actually worked.
Make Your Rating System Clear: Ensure that customers are told what the ratings represent. Each person may think 5 stars means something different until you eliminate that ambiguity.
Companies seem desperate for customer feedback. They bombard us with requests to take surveys, share our thoughts, award stars etc. In theory, all this data should be helpful to the customer community and to the companies asking—but only if the data is accurate. Thanks, Shankar Verdantam and NPR for taking on this issue!
Let’s make the world better with better customer listening!
