Rating Rotten Tomatoes: Is the review-aggregate site getting a fair judgment?

Features
Movies Features

As consumers, we rely on ratings and reviews to discern whether the purchase of a product or service is worth our hard-earned dollars. I cannot imagine buying a book on Amazon or going to a restaurant that I located on Yelp without inspecting its reviews. Movies are no different. Before watching a film at the theatre or even on Netflix, Rotten Tomatoes has been my trusted adviser for years.

Thus, when numerous articles blaming Rotten Tomatoes for poor box-office numbers surfaced on the web last fall after Hollywood’s worst summer in 20 years, without hesitation I jumped on the bandwagon. Last September, The New York Times published an article, “Attacked by Rotten Tomatoes,” citing key Hollywood executives’ hostility towards the movie-rating website. According to the piece, a CEO at a major movie studio made it his personal mission to destroy the movie-rating site.

Not too long after, in October, legendary film director Martin Scorsese wrote an article for The Hollywood Reporter complaining that film-review aggregators such as Rotten Tomatoes “rate a picture the way you'd rate a horse at the racetrack, a restaurant in a Zagat's guide or a household appliance in Consumer Reports.” He added, “These firms and aggregators have set a tone that is hostile to serious filmmakers—even the actual name Rotten Tomatoes is insulting.”

At first, I was convinced by these arguments. But wait a second…

Rotten Tomatoes is owned by Fandango Media, which is 70% owned by NBCUniversal and 30% owned by Time Warner. So the very executives who are criticizing the site work for companies that own it. Which made me wonder: Why would Hollywood have a site that would shoot itself in the foot? And can Rotten Tomatoes be solely blamed for the poor performance of movies? How much of this sentiment is solely anecdotal? Besides, just as a parent would never give up on her child, it’s hard to imagine an executive ever admitting that her film was bad. Thus, Rotten Tomatoes becomes the perfect scapegoat for a Hollywood bomb.

If only we actually had data to see if there is any correlation between Rotten Tomatoes ratings and box-office numbers, wouldn’t that be great? Lucky for us, that data exists. A few months ago, Yves Bergquist, a data scientist from the University of Southern California’s Entertainment Technology Center, had a chance to look at the numbers.

For those of you who took statistic courses back in school, the method Bergguist used was the Pearson Product-Moment Correlation Coefficient (PMCC). The idea of this methodology is to calculate the correlation between two variables, X and Y. If the value results in 1, this indicates a perfect correlation; if the value is -1, this indicates a perfect negative correlation. A value closer to 0 means that there is less correlation. After examining 150 titles for 2017, the result was a PMCC of just 0.12, or 12%, meaning there is basically no correlation. Furthermore, according to the research, contrary to what might be public opinion, Rotten Tomatoes scores have been going up. The median score was 51 in the 2000s and in the 2010s it was 53. Last year, when the article was written, the median score was 71.

This research seems to substantiate some of the inconsistencies we have seen. According to a recent article on Marketwatch, Blade Runner 2049 had all the ingredients worthy of a blockbuster hit, including strong Rotten Tomatoes ratings, but it grossly underperformed at the box office. If Rotten Tomatoes is to blame for poor movie performance, why is it that we never hear an executive thank Rotten Tomatoes for a box-office hit?

After examining the numbers, the attacks on Rotten Tomatoes seem baseless. If Hollywood is struggling to put more butts in seats, there’s a deeper problem at the core. After all, reductionism of movie critiques is nothing new. When the star rating system was introduced in the 1920s and the thumbs-up and down metrics were introduced in the 1980s, many industry experts debated whether these elemental judgments were helping the industry. Rotten Tomatoes is just a 21st-century remix of what happened in the past.

What’s most fascinating about Berqquist’s research is that, according to the math, the audience’s scores and the critics’ scores are becoming increasingly correlated, according to the PMCC value. As of September 2017, when the article was published, the PMCC score was a whopping 0.87, an 87% correlation. So when Hollywood complains about Rotten Tomatoes scores, they are essentially complaining about the audience’s tastes. Nowadays, it’s basically the same thing.

Kevin Hong is the former chief sales officer of Cinema Intelligence and the author of The Outlier Approach: How to Triumph in Your Career as a Noncomformist.