Yelp keeps the details of its review filter algorithm secret, for obvious reasons, but a new study suggests that there are some things about the review and the reviewer that are more likely to trigger the filter.
Michael Luca and Georgios Zervas recently published a working paper, Fake it Till You Make it: Reputation, Competition, and Yelp Review Fraud, that’s based on a study of more than 316,000 Yelp reviews of Boston restaurants. Their study covered more than 3,600 different businesses.
They found that Yelp’s review filter had hidden more than 50,000 of those, or about 16 percent of all the reviews they scraped for the study. And those filtered reviews shared some common characteristics tied both to the review itself, and to the person/account that wrote it.
1- & 5-Star Reviews Are Filtered More
The study found that the ratings of filtered Yelp reviews are more extreme than published reviews. As the chart below shows, the number of filtered 1-star and 5-star reviews is higher than the number of reviews published with those ratings.
Overall, with everything else being equal, 1- and 5-star reviews are about 3 percentage points more likely to be filtered than 3-star reviews.
The study also found that filtered reviews tend to be shorter than published reviews.
Reviewer Characteristics & Filtered Reviews
Beyond the reviews themselves, the study also connects the person/account writing the review to its likelihood for being filtered.
Filtered reviews tend to be more commonly written by accounts that haven’t posted a lot of reviews. In the study, more than 70% of accounts that had only written one review had that review filtered. Furthermore, the more reviews an account writes, the less likely that account is to have its reviews filtered.
One other interesting finding in the study: reviews from users that don’t have a profile image are 41 percentage points more likely to be filtered than accounts that have a profile image.
To a degree, the data confirm what reputation management best practices have suggested for some time now: Yelp reviews written from trusted accounts are more likely to “stick.” And this is why so many local businesses get frustrated when they first ask customers (or friends!) to write positive reviews on Yelp; if those people aren’t already known Yelp users with trusted accounts, their reviews — whether real or fake — are more likely to be filtered.
Speaking Of Fake Reviews…
The study also gets into the issue of fake reviews on Yelp, and draws a few conclusions that are worth mentioning briefly:
Restaurants are more likely to engage in review fraud when they have fewer reviews
Restaurants that have recently received bad reviews are more likely to engage in positive review fraud
Chain restaurants are less likely to leave fake reviews compared to independent restaurants
Businesses with claimed pages are significantly more likely to post fake 5-star reviews
That last point seems counterintuitive, perhaps, but the study notes that this aspect of the data — trying to determine what reviews are fake and which aren’t — is “imperfect” because Yelp admits that its review filter doesn’t catch all fake reviews, and not all filtered reviews are fake.
The full paper can be downloaded in PDF format.
Comments