The Process Of Assigning Games Review Scores
Why numbers and taglines don’t make for an informative, helpful, or representative representation of games media. Published on Mon, Sep 23, 2019. Written by Michael Bassili.
No one is perfect, and criticism is always welcome and expected. — Amitabh Bachchan
It seems the average reviewer of media, both video games or other, gravitates to the use of review scores. PC Gamer, The Needle Drop/Anthony Fantano, Roger Ebert, all assign numerical scores to the media they critique. And, this is fine in the short term. Consumers might want to know if they should spend their hard earned CAD$80 for a new, AAA release, and a review score could help them skip a general impression of the title. Maybe your average consumer doesn’t have a handful of minutes to read a collection of lengthy think-pieces on how the character of Mario makes you feel. They value speed and accuracy. They want to know if a game performs relatively well, has certain desirable features, etc. And, in these cases, scores are helpful. In the short term. In the long-term, I find these numerical assignments pernicious. To see why, I want to talk about a website I don’t particularly like.
If you keep your feathers well oiled the water of criticism will run off as from a duck’s back. — Ellen Swallow Richards
Over the years, IGN has morphed into a website I view in kind of a sour light. Unlike other “reputable” sources, such as Kotaku, Polygon, PC Gamer, and Game Informer, IGN has routinely left a sour taste in my mouth. Look no further than their reviews of the following games…
This was given a 7.7/10 on release. IGN’s own guiding practices, this game lands squarely into the “Good” category. As per their description of their review scores: “Playing a Good game is time well spent. Could it be better? Absolutely. Maybe it lacks ambition, is too repetitive, has a few technical bumps in the road, or is too repetitive, but we came away from it happy nonetheless. We think you will, too.”  This groups the game in with games like Destiny (2014) and LEGO Jurassic World (2015). On its own, this review score seems accurate (who am I to argue against some reviewer’s opinion, as I’m sure many reading my reviews have bones to pick with me).
But, if you inspect ALL of the review scores IGN has given to COD games,  you start to notice a pattern. Fig.1 plots all the IGN review scores of the major COD games. As you can see, all games tend around the 80% score, despite many of the games being objectively worse in quality than the rest. And, that’s not hyperbole or exaggeration. The high-points, which I believe most can assign to MW1, MW2, World At War, and maybe BLOPS, are rated the same as Ghosts (the game where you are a dog for one mission and you run around clicking on enemies to eat). I should note that everyone has a favorite COD game, including the ones I’m insulting here, and that’s fine. We all like what we like. No problem with that.
The issue I’m trying to highlight here is relativism. The concept that these review scores inherently invite comparison over time, only getting worse as time goes on. Sensibilities change, and scores assigned to games are typically either inflated (remembering a game more-highly than it deserves), neutralized (remembering a game as simply “alright”), or deflated (remembering a game negatively). Modern Warfare 1 was the de-facto standard game for that console generation. It defined, inspired, and instructed the design of an entire era of first-person video games. This has only recently been dissolved, by the way. Don’t believe me? Think back to any shooter from the late-2000s to the mid-2010s, and then think back to how MW1 played on launch. To say that Black Ops 3 (2015) or Advanced Warfare (2014) are on the same creative, kinesthetic, or influential plane as MW1 would devalue the impact that game had on the medium. But, unfortunately, that’s what happens. This isn’t just an IGN issue, by the way; any organization that assigns a numerical rating to a product invites the consumer to calculate comparisons between the contemporary and the long-forgotten.
One thing I enjoy about Anthony Fantano’s album reviews is that he assigns culturally significant albums a score of “Classic.” Albums like Plastic Surgery Disasters (1982) and Ziggy Stardust (1972) don’t have an associated review score. I’m not sure if he’s addressed this before, but I suspect he does this because these works are so influential, well-made, and special (in some way) relative to the rest that they must be elevated past numerical assignment of scores, into a place where the big-boys hang out, overlooking the general population. Assigning Thriller (1982) a strong-10 would invite a comparison with contemporary albums Fantano has also rated a 10, such as To Pimp A Butterfly (2015). (One possible reason would be that these “classic” records are all very old, but that prompts me to wonder whether he’ll move contemporary 10s over to the “classic” section in a few decades.)
It’s important to work hard, stay humble, and not let the criticism or the compliments go to your head. — Jessica Sanchez
You’re a games journalist with a family and a dog and stuff. You need to feed all of these mouths. To feed said mouths, you need to write shit about the most recent releases so your dick-head boss doesn’t replace you with an AI. Today, Big Publisher Games released their newest entry: Big Game. The review embargo will be lifted tonight; you’ve have about a few days to play this 100-hour, AAA blockbuster in your spare time. You’re now tasked with assigning it a number out of 10. This number will brand the game on Metacritic, and other aggregate sites. On top of all that, there’s always the looming possibility that (1) you’re bad at playing this game, (2) the game crashed on you a shit-ton, impeding your ability to review it, (3) your opinion differs STRONGLY with those of other, more popular people, or (4) you had little time to actually appreciate the game before writing your review. What do you say about the game?
You say that the game has good graphics, plays alright, has a good soundtrack, and it has both single-player & multi-player components. 8/10.
What I’m getting at here is that the structure of gaming journalism (in the year 2019, long live the empire) facilitates the over-simplicity of reviewing complex video games, often reducing them to their most meaningless, shallow variables such graphics and plot. They’ll often layer some light commentary on top of this surface-level analysis. E.g. New Super Mario Bros 2 (2012) looks pretty good, and it has some good level design, and it sounds neat, and the coins are cool I guess, although there’s no store to spend the coins you collect… 8.5/10, just a solid game. In reality, however, games are complex and dynamic, instilling completely different experiences to completely different people. New Super Mario Bros 2 (2012) is actually a derivative, shallow experience, barely iterating off the THREE NSMB games that came just a few years before it. Why buy this at all when the original NSMB game is cheaper and widely available; NSMB2 presents itself as a glorified level pack, at most, and junk-food at its worst. Here’s the kicker: today, in 2019, common consensus is that NSMB2 is not that good of a game! Reviews at the time of release praised this game, but after people had time to form their own personal opinions about the game years later, they (almost collectively) decided that NSMB2 was an overly-disappointing experience. Reviews are etched into aggregate sites forever; people’s opinions are flexible, ever-changing.
Sandwich every bit of criticism between two layers of praise. — Mary Kay Ash
I’m no exception; my reviews certainly aren’t perfect. But, I’m confident that assigning review scores to the games I choose to cover would hinder impact of my thoughts over time. Should we all collectively ditch review scores? I don’t know. Maybe? Most of the top gaming news outlets don’t have review scores (anymore), like Polygon and Kotaku, so maybe the overall trend will shift towards more thoughtful reviews or something. But, review scores are still the most used method of evaluating a game, just take a look at any Reddit review thread. So, I’m not sure what to think. Personally, I don’t see myself ever assigning scores to games, but that might be easier said than done for other, more “professional” journalists. I’m not under any sort of time crunch. I can choose the games I want to cover, and I can also take my time with these reviews. Maybe I’ll land a job at Kotaku or something and find myself writing a day-one review of COD: Ultra Mega Warfare Deluxe (2025). Who knows…
Here are the sources and references I cited above. A lot of them are links to articles published by places like IGN.
This isn’t exhaustive; you can easily find examples of the weird scoring places like IGN are incentivised to publish. Some prime examples are the 2.5D Mario games and the FIFA/NHL/Madden games.
I want to give a special thanks to Kyle Isaac for his support over on Patreon. If you like what I’m doing and want to support me, consider becoming a Patron. You can read more about it on the support page.