How Are Review Site Star Ratings Calculated? The Answer Is Not What You Think
Published: Dec 1, 2020
Once upon a time, if you wanted to visit the best steakhouse in Dallas, you’d ask a friend or relative for a recommendation. If you’re an old soul, you might still stick to that routine — but most people now prefer to search for “best steakhouse Dallas” and go by whatever review sites or Google recommend.
After all, why rely on a friend’s personal biases, tastes, and limitations when you can consult the "crowd" -- the combined opinions of tens, hundreds or even thousands of people? For most companies, glowing reviews from customers have become the ultimate trust signals.
And what better way to gain an instant understanding of the crowd's collective wisdom than to look at a business' star rating? Many consumers never look further than this number when deciding where to dine, where to travel or what products to buy online -- whether for their home or the company where they work .
But are star ratings what you think they are? Most people assume these ratings represent the average of all the star ratings given by individual reviewers. But this is rarely the case. Ratings are based on machine-learning models and complex algorithms -- not simple arithmetic.
Here's a breakdown of how some of the most popular review sources calculate their ratings:
Google. Did you know that even if a company has earned only five-star Google reviews, the search giant might still assign the business a star rating of, say, 4.8? It's true. Google says it considers “a variety of other signals” in creating ratings besides individual reviews. Google is vague about what this means, but there is speculation that the company uses a complex Bayesian formulato extrapolate what the star rating would look like with a larger sample size.
Amazon. The mammoth retailertakes a myriad of considerations into account in its star ratings to give some reviews greater weight than others. Factors that increase a review's influence include its recency and whether the purchase has been verified. Amazon examines each reviewer's history and tendencies as well; those who consistently leave one-star ratings, for example, are given less say in the aggregate score.
TripAdvisor. The travel resource calls its ratings a “popularity ranking” system. Its algorithm accounts for four factors — recency, quality (based on users’ star ratings -- or "bubble ratings"), quantity (only to ensure the number of reviews is high enough for statistical significance), and consistency (an overall picture of how the other factors link together).
Uber. The ride-hailing service prioritizes recent reviews of its drivers; a driver’s rating is based on the last 500 ratings received. When a rider gives a driver a rating lower than five stars, the app asks for the reason why. If Uber determines that the rider gave the driver a low rating because of price, traffic or other issues beyond the driver's control, that rating is removed from their average.
Scott is founder and CEO of Idea Grove, one of the most forward-looking public relations agencies in the United States. Idea Grove focuses on helping technology companies reach media and buyers, with clients ranging from venture-backed startups to Fortune 100 companies.