Ratings and Reviews: Implementation Details Make All the Difference

Are Ratings and Reviews Worth It?

New data illustrates the power of enabling customers to enter into conversation with each other around purchase decisions. Simply put, everyone selling stuff online should consider adding customer ratings and reviews. Consider the following, lifted from Bryan Eisenberg's ClickZ article:

According to Jupiter Research, 77 percent of online shoppers use reviews and ratings when purchasing.

Reviews drive 21 percent higher purchase satisfaction and 18 percent higher loyalty, according to Foresee Results.

In a study of 2,000 shoppers, 92 percent deemed customer reviews as "extremely" or "very" helpful, finds eTailing Group.

BizRate found 59 percent of users considered customer reviews to be more valuable than expert reviews.

In a CompUSA-iPerceptions study, 63 percent of consumers indicated they're more likely to purchase from a site if it has product ratings and reviews.

86.9 percent of respondents said they'd trust a friend's recommendation over a critic's review, while 83.8 percent said they'd trust user reviews over a critic, according to MarketingSherpa.

92.5 percent of adults said they regularly or occasionally research products online before buying in a store, found BIGresearch.

According to Keller Fay, 63 percent of all WOM is positive. Across all Bazaarvoice clients, 80 percent of products rate 4 or 5 stars out of 5.

So user ratings and reviews are, very often, a good thing for e-commerce sites. But not all ratings and reviews are created equal, and there are better and worse ways to work with them.

 

Ratings Are Meaningless, sort of, and that's OK

I've heard arguments that ratings are bad because they are too subjective. And this attitude is only half wrong. What, for example, do ratings mean? This is far from clear. Is your four-star rating of a movie the same as mine? Clearly not. Ratings are subjective to begin with, and they're especially problematic when you consider that for some users, a five-star rating means "the best thing EVAR" and for others it means "nothing terribly wrong with this thing." When you average out a collection of subjective ratings along a scale whose meaning is subject to a range of biases, you have muddy waters indeed. So yes, ratings are subjective.

Done well, tooltips help. Hover the mouse over three stars and a little popup message says "average." But still, one person's "good" is by no means the same as another person's "good."

There are a couple approaches to increasing the meaningfulness of ratings. One is to use more specific tooltips with a clearer value statement than "good." For example, use labels like "useful," "professional-quality, " or "would recommend."

But ultimately, ratings don't need to be meaningful in the strictest sense to add value. Ratings are, after all, a basis for comparison among like objects. So fixing a quantitative value to an object based on user ratings isn't important--what matters is that you can tell which object is more liked than which other object.

 

Use Multidimensional Ratings for Non-Equivalent Sets of Objects

The more specific the context, the more meaningful ratings can be. The more equivalent the objects are, in other words, the more meaningfully they can be differentiated based on subjective measures like ratings. This is why ratings are so useful for resellers like Amazon.com--if you're like me, you need help deciding between the 17 varieties of 6" non-stick omelette pans, and the opinions of people who've used them are valuable.

This is also why I occasionally discourage clients from simply adding a typical five-star rating system to their product pages. Take the following scenarios:

• A retailer offers a huge range of highly specialized and very similar products--fanny packs, for example. Here the approach is not to create the very best fanny pack, but instead to create the perfect fanny pack for every conceivable need.
• A retailer offers a very limited set of products with complex feature sets, mobile devices for example. Here the approach is to offer the premium product within each of several discrete subcategories.

In each of these cases, simple ratings, comparing the products to each other, adds little or no value. The fanny pack shopper gets no help finding the particular fanny pack to hold her water bottles, energy goo, and cell phone firmly while training for a marathon. The mobile device shopper learns nothing about the premium features and services of each device.

This is where complex, or multidimensional, ratings come in. Instead of rating the product along a single scale, site users rate individual attributes of the product. Here's a very simple example:

Here the overall rating is the average of the user's ratings of individual attributes. The attribute ratings convey some very useful information the overall rating doesn't: That the Corolla, according to this user, looks great but drives crappy. Good to know.

But this example also illustrates some of the challenges of implementing a multidimensional rating system. What, for example, does "quality" mean? How is it separate from the other dimensions being rated?

To implement multidimensional ratings effectively, you need an unambiguous rating taxonomy that applies across all the objects in the set. This is a place where strong user research can help: Understand the way your customers think about and evaluate products, and provide individual attribute ratings that address the specific criteria they use to make choices.

 

Put Reviews to Work

When they first appeared on the web, reviews were typically structured like flat discussion threads. Each new review appeared in chronological order in a list of reviews. The list was more or less fixed to the product detail page, and beyond a certain volume they ceased to be useful. But you can do better than that.

Treat individual reviews as content objects. Attach behavioral metadata to them, and use quality measures to build self-organizing structures. (See my earlier post about quality for more on this.) Surface the best ones alongside detailed product information. Use them on product comparison charts. Run A/B tests to compare conversion across different presentations of reviews.

 

Attach Ratings to Individual Reviews

When users add a review, require them to also add a rating, and display the rating with the review. The typical colored stars are widely understood and instantly provide context to individual reviews, enabling readers to skim long lists of reviews and quickly identify the general gist of each review.

Beyond reading aids, though, ratings can be treated as attributes of individual review objects, providing metadata to aid dynamic presentations.

 

Add an Explicit Layer of Social Metadata

Provide a way for readers to give feedback on individual reviews. Typically this mechanism works through some kind of voting, along the lines of "Was this review helpful? Yes / No." This kind of feedback can help you separate the wheat from the chaff and present the most helpful reviews. In combination with the rating metadata, you can architect powerful presentations of user reviews, along the lines of what Amazon's done recently:

Here the ratings are separated into "favorable" and "critical" groups using the attached ratings, then the reviews from each group deemed most helpful by users are presented together--an instant conversation that provides in-depth, two-sided information about the product.

 

Stay Out of the Tool-Centric Mentality

In the world of social tools, ratings and reviews are about as plug-and-play as it gets. And there are plenty of situations where you can buy a tool like BazaarVoice or PowerReviews, plug it in, and call it good.

But take care to think strategically about what you want to accomplish, and consider whether a custom implementation might serve your needs better. The worst thing you can do is to lock yourself into an approach that fails to take full advantage of the opportunity to add the credibility and persuasiveness of the customer voice to your site.

Do simple ratings make sense on your site, or do you need to consider multidimensional ratings? Can you attach layers of social metadata to your reviews to provide a basis for self-organization among them? How do you present customer reviews alongside product detail information? Alongside excerpted third-party reviews? Do you have an opportunity to leverage the reputations of individual users to provide "expert" voices? What's the relationship in that case between reviews and the identities of their authors?

And beyond even the architecture of ratings and reviews on your site, think about the ongoing maintenance requirements. How will you moderate reviews? What's your policy for appropriate content? What legal considerations do you need to address?

So yes, do add ratings and reviews--just don't let any technology vendors tell you it's as easy as installing something on your server.

Comments

[Leigh Choate - Bazaarvoice]

Hi, Ryan,

This is an insightful post, and of course I agree that ratings and reviews are helpful to shoppers. Bazaarvoice clients have seen all types of business benefits.

I do have to take exception with the "plug and play" reference to Bazaarvoice. While our implementation is easy on our client's site, it's also fully customizable, and each client has a dedicated community manager that works with them to make user-generated content a strategic corporate initiative.

We now see ratings and reviews featured in Best Buy Sunday circular ads, in Sears store signage, on Bath & Body Works emails. Wal-Mart even encourages its millions of customers to check out reviews by printing their review site on every customer receipt.

There are many free and cheap options out there that are just a widget, but that is not how we - and our clients - see it.

[Ryan]

Thanks for the note, Leigh. Sorry, I didn't mean to cast aspersion on BazaarVoice, only to make the point that customization of these kinds of tools is an important, and too-often overlooked, part of a successful implementation.

A lot of folks tend, understandably, to stretch their budgets to buy a tool without understanding how to get the most out of it. Great to hear your program is helping them!

[Ansur]

Ryan,
I completely agree with the 'uselessness' of having a single dimension rating. People evaluate different things differently. It is important to understand what their evaluation criteria is upfront and collect information which matches that criteria. At ReviewScale we are working on developing easy to build forms with multi-dimensional ratings where the scores are calculated automatically and is meaningful.